Patent Description:
With the development of communication technology and miniaturization of electronic devices, personal terminals are being widely used by general consumers. In particular, portable personal terminals such as smart phones or smart tablets have been widely used in recent years. Most of the terminals comprise communication functions. Users can search the internet or exchange messages with other users using terminals.

Further, with the development of miniaturized camera technology, miniaturized microphone technology, miniaturized display technology and miniaturized speaker technology, most of the terminals such as smart phones are being equipped with cameras, microphones, displays and speakers. Users can record a voice or record a video that includes a voice using terminals. Users can check the recorded voice through a speaker included in the terminal or check the recorded video through a display.

Users can transmit the voice that is currently being recorded or the video that is currently being recorded to at least one other user in real time. This at least one other user can check the video or voice that is currently being recorded by the terminal of the other user through a terminal.

<CIT> discloses a method for text-to-speech synthesis with personalized voice. The method includes receiving an incidental audio input of speech in the form of an audio communication from an input speaker and generating a voice dataset for the input speaker. The method includes receiving a text input at the same device as the audio input and synthesizing the text from the text input to synthesized speech including using the voice dataset to personalize the synthesized speech to sound like the input speaker.

<CIT> discloses a method for unsupervised, on-line adaptation in speaker verification. The method comprises detecting a channel of a verification utterance, learning vocal characteristics of the speaker on the detected channel, and transforming the learned vocal characteristics of the speaker from the detected channel to the speaker model of a second channel.

According to the disclosed embodiments, a terminal according to claim <NUM> that is capable of performing real time broadcasting more effectively, and an operating method according to claim <NUM> are provided.

Further, according to the embodiments, a terminal that is capable of expanding human relations through real time broadcasting service, and an operating method thereof may be provided.

An operating method of a terminal that provides a service for performing a real time broadcasting through a broadcasting channel, according to the present invention includes performing the real time broadcasting for guest terminals of a plurality of guests who have entered the broadcasting channel; recognizing a voice of a certain user of the host terminal; recognizing voice characteristics of the certain user from the voice of the certain user when the voice of the certain user is recognized for more than a standard period of time; receiving a first item and a first text, from a first guest terminal of a first guest among the plurality of guests; generating a first voice message converted from the first text into the voice of the certain user based on the voice characteristics of the certain user; and outputting the first voice message through a communication interface of the host terminal.

In some embodiments not covered by the claims, the operating method of the terminal may further include preparing an algorithm for generating the voice message converted from the certain text into the voice of the host.

In some embodiments not covered by the claims, the generating the voice message converted from the certain text into the voice of the host may comprise generating the voice message by applying the voice of the host and the certain text to the algorithm.

In some embodiments not covered by the claims, the preparing the algorithm for generating the voice message converted from the certain text into the voice of the host may comprise preparing a learning model learned regarding a correlation between a plurality of voices, a plurality of texts, and a plurality of voice messages converted from the plurality of texts into the plurality of voices, respectively.

According to the invention, the operating method of the terminal further includes extracting voice characteristics from the voice of the certain user such as the host; generating a comparison voice based on the extracted voice characteristics; comparing the voice of the certain user, such as the host, and the comparison voice; and storing the voice characteristics depending on a result of the comparison.

According to the invention, the comparing the voice of the host and the comparison voice may comprise calculating an error of sampling values between the voice of the host and the comparison voice, and the storing the voice characteristics depending on the result of the comparison may comprise storing the voice characteristics in response to the error being not more than a standard value.

In some embodiments, the generating the first voice message converted from the first text, such as a certain text, into the voice of the certain user, such as the host, comprises generating the voice message based on the certain text and the voice characteristics.

In some embodiments, the at least one or more items may have commodity values within the service.

In some embodiments, the operating method of the terminal may further include directly participating in the broadcasting by a first guest, of the at least one or more guests, who entered the broadcasting channel; and allocating an area other than the area allocated to the host, of the at least two areas of the display, to the first guest.

A terminal according to an embodiment of the present invention includes a display and an input/output interface. The terminal includes a communication interface; a processor; and a memory configured to store instructions executable by the processor, wherein the processor is configured to execute the instructions for performing a real time broadcasting for guest terminals of a plurality of guests who have entered a broadcasting channel; recognizing a voice of a certain user of the host terminal; recognizing voice characteristics of the certain user from the voice of the certain user when the voice of the certain user is recognized for more than a standard period of time; receiving a first item and a first text from a first guest terminal of a first guest among the plurality of guests through the communication interface; generating a first voice message converted from the first text into the voice of the certain user based on the voice characteristics of the certain user, and outputting the first voice message through the communication interface of the host terminal.

In some embodiments not covered by the claims, the processor may prepare a learning model learned regarding a correlation between a plurality of voices, a plurality of texts, and a plurality of voice messages converted from the plurality of texts into the plurality of voices, respectively, and generate the voice message by applying the voice of the host and the certain text to the learning model.

In some embodiments, the terminal may further include a memory that stores the learning model.

According to the invention, the processor extracts voice characteristics from the voice of the certain user, such as the host, generates a comparison voice based on the extracted voice characteristics, compares the voice of the certain user, such as the host, and the comparison voice, calculates an error of sampling values between the voice of the certain user and the comparison voice, storing the voice characteristics depending on a result of the comparison, which comprises storing the voice characteristics in response to the error of the sampling values being not more than a standard value used to determine whether the comparison voice is similar to the voice of the certain user, and depending on a result of the comparison, generates the first voice message based on the certain text and the voice characteristics of the certain user.

In some embodiments, when a first guest, of the at least one or more guests who entered the broadcasting channel, directly participates in the broadcasting, the processor may control the display such that an area other than the area allocated to the host, of the at least two areas of the display, may be allocated to the first guest.

A terminal according to the disclosed embodiments, and an operating method thereof can perform real time broadcasting more effectively.

Further, a terminal according to the disclosed embodiments, and an operating method thereof can expand human relations through real time broadcasting service.

The advantages and characteristics of the present disclosure, and the method for achieving those advantages and characteristics will be clarified with reference to the embodiments that will be described hereinafter together with the drawings attached hereto. However, the present disclosure is not limited to the embodiments disclosed hereinafter, but may be implemented in various different forms, and the present embodiments are provided merely for the purpose of complete disclosure of the present disclosure, and for the purpose of informing a person skilled in the art of the complete scope of the present disclosure, and the present disclosure is to be defined only by the scope of the claims. Like reference numerals indicate like components throughout the entirety of the specification.

Even though "a first" or "a second" and the like are used to describe various components, these components are not limited by the aforementioned terms. The aforementioned terms can only be used to differentiate one component from other components. Therefore, a first component mentioned hereinafter may be a second component within the technical idea of the present disclosure.

The terms used in the present specification were used to describe the embodiments, and not to limit the present disclosure. In the present specification, a singular form includes plural forms unless specially mentioned otherwise. "Comprises" or "comprising" used in the present specification imply that the mentioned component or step does not exclude the presence or addition of one or more other component or step.

Unless defined otherwise, all the terms used in the present specification may be construed to mean what may be commonly understood by a person skilled in the art. Further, the terms defined in generally used dictionaries should not be construed ideally or overly unless clearly defined specially.

<FIG> is a system configuration view illustrating an environment where a terminal according to an embodiment of the present disclosure operates.

Referring to <FIG>, an environment where a plurality of terminals <NUM>~<NUM> operate may include a server <NUM>, and the plurality of terminals <NUM>~<NUM>. For example, the environment where the plurality of terminals <NUM>~<NUM> operate may include at least one or more servers.

Each of the plurality of terminals <NUM>~<NUM> may be connected by medium of the server <NUM>. For convenience description of the present disclosure, three terminals are illustrated in <FIG>, but the number of terminals is not limited to three. Each of the plurality of terminals <NUM>~<NUM> may be embodied as one of a desktop computer, laptop computer, smart phone, smart tablet, smart watch, mobile terminal, digital camera, wearable device, and portable terminal. Each of the plurality of terminals <NUM>~<NUM> may execute a program or an application.

Each of the plurality of terminals <NUM>~<NUM> may be connected to a communication network. Each of the plurality of terminals <NUM>~<NUM> may be connected to each other or may be connected with the server <NUM> through the communication network. Each of the plurality of terminals <NUM>~<NUM> may output data to the interconnected device or receive data from the interconnected device.

The communication network connected to each of the plurality of terminals <NUM>~<NUM> may include a wired communication network, wireless communication network, or composite communication network. The communication network may include a mobile communication network such as <NUM>, LTE, or LTE-A, etc. The communication network may include a wired or wireless communication network such as Wi-Fi, UMTS/GPRS, or Ethernet, etc. The communication network may include a local area communication network such as Magnetic Secure Transmission (MST), Radio Frequency IDentification (RFID), Near Field Communication (NFC), ZigBee, Z-Wave, Bluetooth, Bluetooth Low Energy (BLE), or InfraRed communication (IR), etc. The communication network may include Local Area Network (LAN), Metropolitan Area Network (MAN), or Wide Area Network (WAN), etc..

Various forms of communication sessions may be established between the plurality of terminals <NUM>~<NUM>. For example, the plurality of terminals <NUM>~<NUM> may exchange messages, files, voice data, images or videos with each other. For example, the plurality of terminals <NUM>~<NUM> may perform real time broadcasting using a Transmission Control Protocol (TCP), User Datagram Protocol (UDP), or Web Real-Time Communication (WebRTC), etc..

In some embodiments, in the plurality of terminals <NUM>~<NUM>, an application enabling real time broadcasting to be performed or to be viewed may be installed. A user of the first terminal <NUM>, of the plurality of terminals <NUM>~<NUM>, may generate a broadcasting channel for performing the real time broadcasting through the application.

Further, each user of the second and third terminals <NUM>, <NUM>, of the plurality of terminals <NUM>~<NUM>, may enter the broadcasting channel that the user of the first terminal <NUM> generated, through the application. Each user of the second and third terminal <NUM>, <NUM> may view the broadcasting being conducted by the user of the first terminal <NUM> in real time.

In some embodiments, at least one of the user of the second terminal <NUM> and the user of the third terminal <NUM> may participate in the broadcasting that the user of the first terminal <NUM> generated and conduct the real time broadcasting together. A screen divided into two being displayed on each display of the plurality of terminals <NUM>~<NUM> may each be allocated to the user participating in the broadcasting, of the user of the first terminal <NUM>, the user of the second terminal <NUM>, and the user of the third terminal <NUM>.

The server <NUM> may connect the plurality of terminals <NUM>~<NUM> so that they can perform communication with each other. For example, the server <NUM> may provide a real time broadcasting service so that the plurality of terminals <NUM>~<NUM> can form and participate in a real time broadcasting channel.

One or more exemplary embodiments, terminal may include mobile terminals, electronic devices, cellular phones, smart phones, laptop computers, tablet PCs, electronic book terminals, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigations, MP3 players, digital cameras, etc. However, the terminal is not limited thereto.

<FIG> is a block diagram illustrating a configuration of a terminal according to an embodiment of the present disclosure. Referring to <FIG>, the first terminal <NUM> includes an input/output interface <NUM>, a display <NUM>, a memory <NUM>, a communication interface <NUM>, and a processor <NUM>. Each of the second terminal <NUM> and the third terminal <NUM> illustrated in <FIG> may be embodied similarly or identically as the first terminal <NUM>.

The input/output interface <NUM> may receive signals from outside. The input/output interface <NUM> may receive the signals from a user of the first terminal <NUM>. Further, the input/output interface <NUM> may receive the signals from an external device. The input/output interface <NUM> may include, for example, a microphone, camera, keyboard, mouse, trackball, touchscreen, button, switch, sensor, network interface, or other input devices, etc. The input/output interface <NUM> may receive voices from outside through the microphone included in the input/output interface <NUM>.

Further, the input/output interface <NUM> may receive images or videos recorded from the camera (not illustrated) included in the input/output interface <NUM>, or receive gestures from the user of the terminal <NUM>.

The input/output interface <NUM> may include a display <NUM>. For example, the display <NUM> may include a flat panel display device such as Liquid Crystal Display (LCD), Organic Light Emitting Diode (OLED), or Plasma Display Panel (PDP), etc. The display <NUM> may include a curved-surface display or a flexible display. The display <NUM> may include a touchscreen. In a case where the display <NUM> includes the touchscreen, the display <NUM> may receive a touch input from the user of the first terminal <NUM>.

The display <NUM> may display data. Further, the display <NUM> may display a result of computation performed by the processor <NUM>. Further, the display <NUM> may display data stored in the memory <NUM>. The display <NUM> may display the data received through the input/output interface <NUM> or display the data received by the communication interface <NUM>.

In some embodiments, in a case where the real time broadcasting application is being executed in the first terminal <NUM>, the display <NUM> may output an image of the user of the first terminal <NUM>. Further, in a case where an image of the user of the second terminal <NUM> or an image of the user of the third terminal <NUM> is being received through the communication interface <NUM>, the display <NUM> may output the image of the user of the second terminal <NUM> or the image of the user of the third terminal <NUM> together with the image of the user of the first terminal <NUM>.

In some embodiments, the display <NUM> may receive a certain input from the user of the first terminal <NUM>. The certain input may be an input of selecting one item of at least one or more items or an input of entering a certain text. For example, the item may have a commodity value within the application. Users of the application may purchase the item, and present the purchased item to each other as a gift.

In some embodiments, in a case where the real time broadcasting application is being executed in the first terminal <NUM>, the input/output interface <NUM> may output sound. The input/output interface <NUM> may output the sound received through the input/output interface <NUM> or the sound received from the second terminal <NUM> or the third terminal <NUM> through the communication interface <NUM>. For example, the input/output interface <NUM> may include a speaker (not illustrated).

In some embodiments, in a case where the real time broadcasting application is being executed in the first terminal <NUM>, the input/output interface <NUM> may receive profile information received from the user of the first terminal <NUM> or receive a user input. For example, the profile information of the user may include at least one of a photograph, hobby information, gender information, country information and age information of the user of the terminal <NUM>. Further, the profile information of the user may further include a video recorded by the user. In addition, the user input may be a touch input being received from the user of the terminal <NUM>.

The memory <NUM> may store data. The memory <NUM> may store voice data, image data or profile information of the user received from the input/output interface <NUM>. In addition, the memory <NUM> may store a result of computation performed by the processor <NUM>. For example, the memory <NUM> may store a voice encoded by the processor <NUM>. The memory <NUM> may store the data to be output to outside through the communication interface <NUM> or store the data received from outside through the communication interface <NUM>.

The memory <NUM> may store a software or program. For example, the memory <NUM> may store programs and various types of data such as applications and application programming interfaces (API), etc. The memory <NUM> may store instructions that are executable by the processor <NUM>.

The memory <NUM> may include at least one of a volatile memory or nonvolatile memory. The memory <NUM> may include at least one of, for example, flash memory, Read Only Memory (ROM), Electrically Erasable ROM (EEROM), Erasable Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), Hard Disk Drive (HDD) or register. The memory <NUM> may include, for example, a file system, database, or embedded database, etc..

The communication interface <NUM> may output data to outside of the terminal <NUM> or receive data from the outside. The communication interface <NUM> may output the data to the server <NUM> or to an external device. The communication interface <NUM> may receive the data from the server <NUM> and from the external device. The communication interface <NUM> may output a result of computation performed by the processor <NUM> to outside.

In some embodiments, in a case where the real time broadcasting application is executed in the first terminal <NUM>, the communication interface <NUM> may receive an image or voice from the second terminal <NUM> or the third terminal <NUM>.

Further, the communication interface <NUM> may transmit an item or a certain text selected by the user of the first terminal <NUM> to the second terminal <NUM> or the third terminal <NUM>. Further, the communication interface <NUM> may receive an item or a certain text from the second terminal <NUM> or the third terminal <NUM>.

The communication interface <NUM> may include a remote network interface such as <NUM> module, LTE module, LTE-A module, Wi-Fi module, WiGig module, Ultra Wide Band (UWB) module, and LAN card, etc. Further, the communication interface <NUM> may include a local network interface such as Magnetic Secure Transmission (MST) module, Bluetooth module, Near Field Communication (NFC) module, Radio Frequency IDentification (RFID) module, ZigBee module, Z-Wave module, and InfraRed communication (IR), etc. Further, the communication interface <NUM> may include other network interfaces.

The processor <NUM> or each of the components included in the processor <NUM> may be embodied in the form of software or hardware. For example, the software may be embodied as program execution instructions such as mechanical codes, firmware codes, embedded codes, and applications, etc. The hardware may be an electric electronic circuit, processor, computer, pressure sensor, inertia sensor, MicroElectroMechanical System (MEMS), passive elements, or a combination thereof.

The processor <NUM> may control operations of the terminal <NUM>. The processor <NUM> may be interconnected with each of the components included in the terminal <NUM>, and control operations of each component included in the terminal <NUM>. The processor <NUM> may control the operations of the terminal <NUM> in response to the signal received by the input/output interface <NUM>.

In some embodiments, in a case where the first terminal <NUM> executes the real time broadcasting application as a host terminal, the processor <NUM> may recognize the voice of the user of the first terminal <NUM> being received through the input/output interface <NUM>. Further, in a case where a certain text is being received through the communication interface <NUM>, the processor <NUM> may prepare to generate a voice message converted from a certain text into a voice of the user of the first terminal <NUM>.

In some embodiments, the processor <NUM> may generate a voice message converted from a certain text into a voice of the user of the first terminal <NUM> using a prepared learning model. In other embodiments, the processor <NUM> may extract characteristics from the voice of the user of the first terminal <NUM>, and generate the voice message converted from the certain text into the voice of the user of the first terminal <NUM> using the extracted characteristics.

Detailed operation methods of the plurality of terminals <NUM>~<NUM> and the server <NUM> may be described with reference to <FIG>.

<FIG> is a view illustrating a method for executing a real time broadcasting application in a terminal according to an embodiment of the present disclosure.

Referring to <FIG>, the user of the first terminal <NUM> may execute the real time broadcasting application. The user of the first terminal <NUM> may generate a broadcasting channel through the real time broadcasting application. The user of the first terminal <NUM> may perform a voice broadcasting or a video broadcasting in real time through the broadcasting channel.

In some embodiments, when the user of the first terminal <NUM> generates the broadcasting channel and enters the broadcasting channel, the display <NUM> of the first terminal <NUM> may be divided into at least two areas <NUM>, <NUM>. The first area <NUM>, of the at least two areas <NUM>, <NUM>, may be allocated to the user of the first terminal <NUM>.

In some embodiments, in a case where the user of the first terminal <NUM> performs a voice broadcasting, a profile photograph set by the user of the first terminal <NUM> may be displayed in the first area <NUM>. In a case where the user of the first terminal <NUM> performs a video broadcasting, the video that the user of the first terminal <NUM> records may be displayed in the first area <NUM>.

In some embodiments, the user of the second terminal <NUM> and the user of the third terminal <NUM> may enter the broadcasting channel that the user of the first terminal <NUM> generated. The user of the second terminal <NUM> and the user of the third terminal <NUM> may watch the broadcasting that the user of the first terminal <NUM> performs, as guests.

In some embodiments, at least one of the user of the second terminal <NUM> and the user of the third terminal <NUM>, may directly participate in the broadcasting. If the user of the second terminal <NUM> directly participates in the broadcasting, the second area <NUM>, of the at least two areas <NUM>, <NUM>, may be allocated to the user of the second terminal <NUM>.

In some embodiments, in a case where the user of the second terminal <NUM> performs a voice broadcasting, a profile photograph that the user of the second terminal <NUM> set may be displayed in the second area <NUM>. If the user of the second terminal <NUM> performs a video broadcasting, the video that the user of the second terminal <NUM> records may be displayed in the second area <NUM>.

If the user of the second terminal <NUM> directly participates in a broadcasting, the user of the first terminal <NUM> and the user of the second terminal <NUM> may conduct the broadcasting together. In addition, the user of the third terminal <NUM> may watch the broadcasting being conducted by the user of the first terminal <NUM> and the user of the second terminal <NUM>.

<FIG> is a view illustrating a method for executing a real time broadcasting application in a terminal according to another embodiment of the present disclosure.

Referring to <FIG> and <FIG>, the user of the second terminal <NUM> or the user of the third terminal <NUM> may present an item to the user of the first terminal <NUM> during broadcasting. For example, the item may have a commodity value within the application. Users of the application may purchase the item, and present the purchased item to each other as a gift.

In some embodiments, the user of the second terminal <NUM> or the user of the third terminal <NUM> may touch an item gift icon <NUM>. The item gift icon <NUM> may be displayed in a partial area of the display. When the user of the second terminal <NUM> or the user of the third terminal <NUM> selects the item gift icon <NUM>, an icon popup window <NUM> may be displayed.

In some embodiments, in the icon popup window <NUM>, at least one or more items <NUM>~<NUM> may be displayed, and the user of the second terminal <NUM> or the user of the third terminal <NUM> may select one of the at least one or more items <NUM>~<NUM>. For example, each of the at least one or more items <NUM>~<NUM> may have different commodity values from one another.

The user of the second terminal <NUM> or the user of the third terminal <NUM> may select one of the at least one or more items <NUM>~<NUM>. In some embodiments, the user of the second terminal <NUM> or the user of the third terminal <NUM> may transmit a certain text together with the selected item. For example, the user of the second terminal <NUM> or the user of the third terminal <NUM> may enter a text saying 'nice to meet you'. The user of the second terminal <NUM> or the user of the third terminal <NUM> may transmit the message saying 'nice to meet you' to the user of the first terminal <NUM> together with the selected item.

In other embodiments, the user of the first terminal <NUM> or the user of the third terminal <NUM> may present an item to the user of the second terminal <NUM> as a gift during broadcasting. The user of the second terminal <NUM> or the user of the third terminal <NUM> may select one of the at least one or more items <NUM>~<NUM>, and the user of the first terminal <NUM> or the user of the third terminal <NUM> may transmit a certain text together with the selected item.

Referring to <FIG>, the certain text transmitted to the user of the first terminal <NUM> or the user of the second terminal together with the selected item may be converted into a voice message and be output.

In some embodiments, the certain text may be converted into a voice message using the voice of a certain user. More specifically, in a case of a certain text that the user of the second terminal <NUM> or the user of one of the third terminals <NUM> transmitted to the user of the first terminal <NUM>, the certain text may be converted into a voice message using the voice of the user of the first terminal <NUM>.

Otherwise, in a case of a certain text that the user of the first terminal <NUM> or the user of one of the third terminals <NUM> transmitted to the user of the second terminal <NUM>, the certain text may be converted into a voice message using the voice of the user of the second terminal <NUM>.

Otherwise, the certain text may be converted into a voice message using the voice of the user who transmitted the certain text. That is, in a case of a certain text that the user of the second terminal <NUM> transmitted to the user of the first terminal <NUM>, the certain text may be converted into a voice message using the voice of the user of the second terminal <NUM>.

Referring to <FIG>, operations of generating a voice message using a certain text and the voice of a certain user may be performed in the processor <NUM> of the first terminal <NUM> or the second terminal <NUM>. In examples not covered by the claims, the processor <NUM> may be generate the voice message using the prepared learning model. The method in which the processor <NUM> generates the voice message using the prepared learning model may be described with reference to <FIG>.

In other embodiments, the processor <NUM> extracts characteristics of a certain voice, and generates a voice message using the extracted characteristics. The method in which the processor <NUM> generates a voice message using the characteristics of a certain voice may be described with reference to <FIG>.

In some embodiments, in a case of converting a certain text into a voice message using the voice of the user of the first terminal <NUM>, the conversion may be performed in the processor <NUM> of the first terminal <NUM>. In addition, the generated voice message may be transmitted to the second terminal <NUM> and the third terminal <NUM>.

In other embodiments, in a case of converting a certain text into a voice message using the voice of the user of the second terminal <NUM>, the conversion may be performed in the processor <NUM> of the second terminal <NUM>. In addition, the generated voice message may be transmitted to the first terminal <NUM> and the third terminal <NUM>.

<FIG> is a flowchart for illustrating a method for converting a text into a voice message in a terminal according to an example not covered by the claims.

Referring to <FIG>, at step S110, the user of the first terminal <NUM> may open a real time broadcasting channel, and start broadcasting. More specifically, the user of the first terminal <NUM> may generate the broadcasting channel through a real time broadcasting application. The user of the first terminal <NUM> may perform a voice broadcasting or a video broadcasting in real time through the broadcasting channel.

At step S120, the processor <NUM> of the first terminal <NUM> may recognize the voice of a certain user. For example, the certain user may be the user of the first terminal <NUM>. In some embodiments, the processor <NUM> of the first terminal <NUM> may recognize the voice of the user of the first terminal <NUM> being received in the input/output interface <NUM> during broadcasting. More specifically, of the audio data being input into the input/output interface <NUM>, the processor <NUM> may recognize the voice of the user of the first terminal <NUM>, and extract the recognized voice of the user of the first terminal <NUM>.

At step S130, the first terminal <NUM> may receive an item and a certain text from a guest who entered the real time broadcasting channel that the user of the first terminal <NUM> opened in real time through the communication interface <NUM>. In some embodiments, at least one or more guests may participate in the real time broadcasting channel, and an item and a certain text may be received from a certain guest of the at least one or more guests. The received item and the certain text may be transmitted to the processor <NUM>.

At step S140, the processor <NUM> of the first terminal <NUM> may prepare an algorithm for generating a voice message converted from a certain text into the voice of a certain user. For example, the prepared algorithm may be a data recognition model being used to convert the certain text into the voice message using the voice of the certain user. The data recognition model may be a model that is based on a neural network. For example, the learning models such as a Deep Neural Network (DNN), Recurrent Neural Network (RNN), and Bidirectional Recurrent Deep Neural Network (BRDNN) may be used as the data recognition model, but there is no limitation thereto.

The prepared learning model may be a learning model for generating a voice message converted from a certain text into a certain voice. The learning model for generating the voice message may be a result of learning a correlation regarding a plurality of voices, a plurality of texts, and voice messages converted from the plurality of texts into the plurality of voices, respectively.

For example, the processor <NUM> of the first terminal <NUM> may learn a correlation between a certain voice, a certain text, and a voice message converted from the certain text into the certain voice. The terminal <NUM> may exercise an artificial neural network based on the result of learning, and generate a learning model.

In another example, the terminal <NUM> may receive a learning model for generating a voice message from the server <NUM>. In this case, the server <NUM> may generate a learning model that learned the correlation between a certain voice, a certain text, and a voice message converted from the certain text into the certain voice, and provide an application that includes the generated learning model to the terminal <NUM>.

At step S150, the processor <NUM> of the first terminal <NUM> may generate a voice message using the algorithm. More specifically, the processor <NUM> of the first terminal <NUM> may apply the voice of a certain user and a certain text to the algorithm and generate the voice message. The voice message may be a result of the certain text converted into the voice of the certain user.

At step S160, the first terminal <NUM> may output the generated voice message. More specifically, the first terminal <NUM> may output the voice message through the input/output interface <NUM>. Otherwise, the first terminal <NUM> may output the voice message through the communication interface <NUM>.

<FIG> is a flowchart illustrating a method for converting a text into a voice message in a terminal according to an embodiment of the present invention.

Referring to <FIG> and <FIG>, at step S210, the user of the first terminal <NUM> may open a real time broadcasting channel, and start broadcasting. More specifically, the broadcasting channel may be generated through a real time broadcasting application. The user of the first terminal <NUM> may perform a voice broadcasting or a video broadcasting in real time through the broadcasting channel.

At step S220, the processor <NUM> of the first terminal <NUM> recognizes the voice of a certain user. For example, the certain user is the user of the first terminal <NUM>. More specifically, the processor <NUM> of the first terminal <NUM> recognizes the voice of the user of the first terminal <NUM> being received in the input/output interface <NUM> during broadcasting. Further, the voice of the user of the first terminal <NUM> is stored in the memory <NUM>.

At step S230, when the voice of a certain user is recognized for or more than a standard period of time, the processor <NUM> of the first terminal <NUM> extracts characteristics of the voice. For example, the characteristics of the voice may mean a voice-specific accent, frequency brand, formant and pitch, etc. That is, voice characteristics may mean the voice-specific characteristics that can form the corresponding voice.

At step S240, the processor <NUM> of the first terminal <NUM> generates a comparison voice based on the extracted voice characteristics. In addition, at step S250, the processor <NUM> of the first terminal <NUM> compares the voice of a certain user and the generated comparison voice.

At step S260, the processor <NUM> of the first terminal <NUM> stores the voice characteristics in the memory <NUM> depending on the result of comparison. When an error between the voice of the certain user and the comparison voice is not more than a standard value, the processor <NUM> stores the voice characteristics in the memory. For example, the error may be calculated through a difference of sampling values between the voice of the certain user and the comparison voice. The method for calculating the error between the voice of a certain user and the comparison voice is not limited thereto, and may be calculated using various methods.

In some embodiments, in a case where a noise is recognized as the voice of the user of the first terminal <NUM> during voice recognition, the comparison voice generated using the extracted voice characteristics may have a large error compared to the voice of the user of the first terminal <NUM>. Therefore, in order to generate a voice that is similar to the voice of the user of the first terminal <NUM> using the voice characteristics, the processor <NUM> may perform a process of generating the comparison voice using the voice characteristics, and comparing the voice of the user of the first terminal <NUM> and the comparison voice.

At step S270, the first terminal <NUM> receives an item and a certain text from a guest who entered the real time broadcasting channel that the real time user of the first terminal <NUM> opened through the communication interface <NUM>. In some embodiments, at least one or more guests may participate in the real time broadcasting channel, and an item and a certain text may be received from a certain guest of these guests. The received item and the certain text may be transmitted to the processor <NUM>.

At step S280, the processor <NUM> of the first terminal <NUM> generates a voice message based on the certain text and the voice characteristics and output the generated voice message. In some embodiments, the voice message that the processor <NUM> generated based on the voice characteristics may be similar or identical to the voice of a certain user. The first terminal <NUM> may output the voice message through the input/output interface <NUM>. The first terminal <NUM> outputs the voice message through the communication interface <NUM>.

<FIG> is a view illustrating the processor of a terminal according to an embodiment of the present disclosure.

Referring to <FIG>, <FIG> and <FIG>, the processor <NUM> of the first terminal <NUM> may include a voice recognition unit <NUM> and a model application unit <NUM>. Each of the second terminal <NUM> and the third terminal <NUM> illustrated in <FIG> may be embodied similarly or identically as the first terminal <NUM>.

The voice recognition unit <NUM> may recognize the voice of the user of the first terminal <NUM> of the audio data being input into the input/output interface <NUM> of the first terminal <NUM> during broadcasting, and extract the recognized voice. In some embodiments, the voice recognition unit <NUM> may analyze the input audio data and distinguish the input audio data into a voice section and a non-voice section. The voice recognition unit <NUM> may exclude the audio data included in the non-voice section, and recognize the voice of the audio data included in the voice section, and transmit the recognized voice to the model application unit <NUM>.

The model application unit <NUM> may apply the voice of the user of the first terminal <NUM> and the certain text received from outside to the algorithm and generate a voice message. In some embodiments, the learning model for generating the voice message may be a result of learning the correlation between a certain voice, a certain text and a voice message converted from the certain text into the certain voice.

<FIG> is a view illustrating a processor of a terminal according to another embodiment of the present disclosure.

Referring to <FIG>, <FIG> and <FIG>, the processor <NUM> of the first terminal <NUM> may include a voice recognition unit <NUM>, a characteristics extraction unit <NUM>, a comparison unit <NUM>, and a voice message creation unit <NUM>. Each of the second terminal <NUM> and the third terminal <NUM> illustrated in <FIG> may be embodied similarly or identically as the first terminal <NUM>.

The voice recognition unit <NUM> illustrated in <FIG> may operate similarly or identically as the voice recognition unit <NUM> illustrated in <FIG>. The voice recognition unit <NUM> may recognize the voice of the audio data included in the voice section, and transmit the recognized voice to the characteristics extraction unit <NUM>.

The characteristics extraction unit <NUM> may extract the characteristics of the voice of the user of the first terminal <NUM>. For example, the voice characteristics may mean the voice-specific accent, frequency band, formant and pitch, etc. That is, the voice characteristics may mean the voice-specific characteristics that can make the corresponding voice. The characteristics extraction unit <NUM> may generate a comparison voice using the extracted voice characteristics.

The comparison unit <NUM> may compare the voice of the user of the first terminal <NUM> and the comparison voice. The comparison unit <NUM> may store the voice characteristics in the memory <NUM> depending on the result of comparison.

In some embodiments, when the error between the voice of the user of the first terminal <NUM> and the comparison voice is not more than the standard value, the processor <NUM> may store the voice characteristics in the memory <NUM>, and transmit the voice characteristics to the voice message creation unit <NUM>.

For example, the error may be calculated through a difference of sampling values between the voice of the user of the first terminal <NUM> and the comparison voice. The method for calculating the error between the voice of the user of the first terminal <NUM> and the comparison voice is not limited thereto, and various methods may be used in the calculation.

If the error between the voice of the user of the first terminal <NUM> and the comparison voice exceeds the standard value, the comparison unit <NUM> may transmit a feedback signal to the characteristics extraction unit <NUM>. When the feedback signal is received in the characteristics extraction unit <NUM>, the characteristics extraction unit <NUM> may extract the characteristics from the voice of the user of the first terminal <NUM> once again.

The voice message creation unit <NUM> may generate a voice message based on a certain text and the voice characteristics, and output the generated voice message.

Referring to <FIG>, each of the plurality of terminals <NUM>~<NUM> according to an embodiment of the present disclosure may perform real time broadcasting more effectively.

Further, each of the plurality of terminals <NUM>~<NUM> may provide a service for expanding human relations through a real time broadcasting service.

The embodiments described above may also be embodied in a recording medium form that includes instructions executable by a computer such as a program module executable by a computer. The computer readable media may be any available media accessible by a computer, and may include all volatile and non-volatile media, and separable and non-separable media.

Further, the computer readable media may include computer storage media or communication media. The computer storage media may include all volatile and non-volatile, and separable and non-separable media embodied in any method or technology for storage of information such as computer readable instructions, data structures and program modules or other data. The communication media may include computer readable instructions, data structures, program modules, or other data of modulated data signals such as carrier waves, or other output mechanisms, and may include any information delivery media.

Claim 1:
An operating method of a host terminal (<NUM>) that provides a service for performing a real time broadcasting through a broadcasting channel, the operating method comprising:
performing (S210) the real time broadcasting for guest terminals (<NUM>, <NUM>) of a plurality of guests who have entered the broadcasting channel;
recognizing (S220) a voice of a certain user of the host terminal;
recognizing (S230-S260) voice characteristics of the certain user from the voice of the certain user when the voice of the certain user is recognized for more than a standard period of time;
receiving (S270) a first item and a first text, from a first guest terminal (<NUM>, <NUM>) of a first guest among the plurality of guests;
generating (S280) a first voice message converted from the first text into the voice of the certain user based on the voice characteristics of the certain user; and
outputting the first voice message through a communication interface (<NUM>) of the host terminal (<NUM>);
wherein the recognizing the voice characteristics of the certain user comprises:
extracting (S230) the voice characteristics from the voice of the certain user;
generating (S240) a comparison voice based on the extracted voice characteristics;
comparing (S250) the voice of the certain user and the comparison voice, wherein the comparing (S250) the voice of the certain user and the comparison voice comprises calculating an error of sampling values between the voice of the certain user and the comparison voice; and
storing (S260) the voice characteristics depending on a result of the comparison, wherein storing (S260) the voice characteristics depending on the result of the comparison comprises storing the voice characteristics in response to the error of the sampling values being not more than a standard value used to determine whether the comparison voice is similar to the voice of the certain user.