Patent Description:
In recent years, a user has played a video game while having voice chat with other users at distant locations who are playing the video game together with the user or watching the moving image depicting the situation in the video game, for example.

Previously proposed arrangements are disclosed in <CIT> and <CIT>.

Some users want to grasp the content of voice chat as text that is obtained as a result of voice recognition on voice in the voice chat. It is desired that such users can get text obtained as a result of voice recognition on voice in voice chat.

However, there are users who do not need text obtained as a result of voice recognition on voice in voice chat such as users who do not want to grasp the text. Providing the text to such users only unnecessarily increases the data traffic.

The present invention has been made in view of the above-mentioned circumstances, and has an object to provide a voice chat apparatus, a voice chat method, and a program that achieve appropriate control on whether or not to provide text obtained as a result of voice recognition on voice in voice chat.

This invention is defined by independent claims <NUM>, <NUM> and <NUM>. Further respective aspects and features of the disclosure are defined in the dependent claims.

<FIG> is a diagram illustrating an exemplary overall configuration of a computer network according to an embodiment of the present invention. As illustrated in <FIG>, voice chat systems <NUM> (<NUM>-<NUM>, <NUM>-<NUM>,. , and <NUM>-n), a voice agent server <NUM>, and a management server <NUM>, each of which mainly includes a computer, are connected to a computer network <NUM> such as the Internet. The voice chat systems <NUM>, the voice agent server <NUM>, and the management server <NUM> can communicate with each other.

The management server <NUM> is, for example, a computer such as a server configured to manage account information regarding users who use the voice chat systems <NUM>. The management server <NUM> stores a plurality of account data associated with the respective users, for example. The account data includes, for example, a user identification (ID) that is identification information regarding the user, real name data indicating the real name of the user, and email address data indicating the email address of the user.

The voice agent server <NUM> of the present embodiment is a server computer such as a server configured to generate text indicating the result of voice recognition processing on received voice, and to generate voice indicating the result of voice synthesis processing on received text, for example. The voice agent server <NUM> may implement a voice recognition engine configured to generate text indicating the result of voice recognition processing on received voice, and a voice synthesis engine configured to generate voice indicating the result of voice synthesis processing on received text.

As illustrated in <FIG>, the voice chat system <NUM> includes a voice chat apparatus <NUM> and a router <NUM>.

The voice chat apparatus <NUM> is a computer capable of inputting or outputting voice in voice chat, such as a video game console, a portable video game apparatus, a smartphone, or a personal computer.

As illustrated in <FIG>, the voice chat apparatus <NUM> includes, for example, a processor 20a, a storage unit 20b, a communication unit 20c, a display unit 20d, an operation unit 20e, a microphone 20f, a speaker <NUM>, and an encoding/decoding unit <NUM>. Note that, the voice chat apparatus <NUM> may include a camera.

The processor 20a is, for example, a program control device such as a central processing unit (CPU), and executes various types of information processing on the basis of programs stored in the storage unit 20b.

The storage unit 20b is, for example, a storage element such as a read-only memory (ROM) or a random access memory (RAM) or a hard disk drive.

The communication unit 20c is, for example, a communication interface for transmitting/receiving data to/from the computers such as the other voice chat systems <NUM>, the voice agent server <NUM>, or the management server <NUM> via the router <NUM> and the computer network <NUM>.

The display unit 20d is, for example, a liquid crystal display, and displays screens generated by the processor 20a or moving images indicated by moving image data received via the communication unit 20c.

The operation unit 20e is, for example, an operation member for operation input to the processor 20a. Note that, the operation unit 20e may be a video game controller.

The microphone 20f is, for example, a voice input device that is used for voice input in voice chat.

The speaker <NUM> is, for example, a voice output device that is used for voice output in voice chat.

The encoding/decoding unit <NUM> includes an encoder and a decoder, for example. The encoding/decoding unit <NUM> encodes input voice to generate voice data indicating the voice. Further, the encoding/decoding unit <NUM> decodes input voice data to output the voice indicated by the voice data.

Further, by executing predetermined pairing processing, as illustrated in <FIG>, an auxiliary apparatus <NUM> configured to assist voice chat can be added to the voice chat system <NUM> according to the present embodiment in addition to the voice chat apparatus <NUM>.

The auxiliary apparatus <NUM> according to the invention as claimed is a portable device, for example, a portable computer such as a smartphone or a tablet device. Note that, the auxiliary apparatus <NUM> may be a stationary computer.

The auxiliary apparatus <NUM> according to the present embodiment includes, for example, a processor 24a, a storage unit 24b, a communication unit 24c, and a touch panel 24d.

The processor 24a is, for example, a program control device such as a CPU, and executes various types of information processing on the basis of programs stored in the storage unit 24b.

The storage unit 24b is, for example, a storage element such as a ROM or a RAM or a hard disk drive.

The communication unit 24c is, for example, a communication interface for transmitting/receiving data to/from the computers such as the voice chat apparatus <NUM> via the router <NUM>. Note that, the communication unit 24c may transmit/receive data to/from the computers such as the other voice chat systems <NUM>, the voice agent server <NUM>, or the management server <NUM> via the router <NUM> and the computer network <NUM>, for example.

The touch panel 24d includes, for example, a touch sensor and a display, such as a liquid crystal display, that are integrated with each other. The touch panel 24d displays screens generated by the processor 24a. Further, the user performs various types of operation on the touch panel 24d, for example, tapping the touch panel 24d, thereby being capable of performing operation input to the processor 24a.

The voice chat apparatus <NUM> and the auxiliary apparatus <NUM> are connected to the router <NUM>, which is connected to the computer network <NUM>, with cables or wirelessly. The voice chat apparatus <NUM> and the auxiliary apparatus <NUM> communicate with the other voice chat systems <NUM>, the voice agent server <NUM>, or the management server <NUM> via the router <NUM>.

In the present embodiment, the plurality of voice chat systems <NUM> (<NUM>-<NUM> to <NUM>-n) support voice chat. Thus, the present embodiment allows the plurality of users using the respective voice chat systems <NUM> to enjoy voice chat. Here, for example, the users may have voice chat while sharing a moving image depicting the situation in a video game that some or all of the users participating in the voice chat are playing.

In the present embodiment, a plurality of users participating in voice chat belong to a group called "party. " Further, the user of the voice chat system <NUM> according to the present embodiment performs predetermined operation, thereby being capable of creating a new party or participating in an already created party.

Further, in the present embodiment, the user of the voice chat system <NUM> in which the auxiliary apparatus <NUM> and the voice chat apparatus <NUM> have been paired with each other performs predetermined operation, thereby being capable of using a voice chat assistance service in the voice chat system <NUM>.

In the voice chat system <NUM> in which the voice chat assistance service is available, the result of voice recognition on voice in voice chat can be displayed on the touch panel 24d of the auxiliary apparatus <NUM> or text can be input for voice chat instead of voice. Further, the user using the voice chat assistance service performs predetermined operation, thereby being capable of stopping using the voice chat assistance service.

In the present embodiment, information associated with parties is managed with party management data exemplified in <FIG>. The party management data is stored in the management server <NUM>, for example. As illustrated in <FIG>, the party management data includes a party ID that is identification information regarding a party and user data associated with users participating in the party. The user data includes user IDs, connection destination address data, type data, assistance service use flags, and the like.

The user ID is, for example, identification information regarding the user. The connection destination address data is, for example, data indicating the address of the voice chat apparatus <NUM> used by the user. The type data is, for example, data indicating the type of the voice chat apparatus <NUM> used by the user. The assistance service use flag is, for example, a flag indicating whether or not the voice chat assistance service is available in the voice chat system <NUM> used by the user. Here, for example, in a case where the voice chat assistance service is available in the voice chat system <NUM>, an assistance service use flag with a value of <NUM> is set. Further, for example, in a case where the voice chat assistance service is unavailable in the voice chat system <NUM>, an assistance service use flag with a value of <NUM> is set.

<FIG> exemplifies the party management data in which the party in which the five users are participating has the party ID of <NUM>. The party management data illustrated in <FIG> includes the five pieces of user data associated with the respective users participating in the party. In the following, the user having the user ID of aaa, the user having the user ID of bbb, the user having the user ID of ccc, the user having the user ID of ddd, and the user having the user ID of eee are referred to as "user A," "user B," "user C," "user D," and "user E," respectively. Further, the user A, the user B, the user C, the user D, and the user E use the respective voice chat systems <NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>, and <NUM>-<NUM>. Further, the voice chat systems <NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>, and <NUM>-<NUM> include respective voice chat apparatus <NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>, <NUM>, and <NUM>-<NUM>.

The party management data exemplified in <FIG> indicates that the voice chat assistance service is available in none of the voice chat systems <NUM>.

Further, in the present embodiment, a copy of the party management data stored in the management server <NUM> is transmitted to the voice chat apparatus <NUM> used by the users participating in the party associated with the party management data. The storage units 20b of the voice chat apparatus <NUM> store the copy of the party management data stored in the management server <NUM>. Thus, the voice chat apparatus <NUM> used by the users participating in the party can identify the addresses of the voice chat apparatus <NUM> used by the users participating in the party.

Further, in the present embodiment, the party management data stored in the management server <NUM> is updated when the user performs operation to participate in the party, operation to start using the voice chat assistance service, or operation to stop using the voice chat assistance service, for example. Every time the party management data stored in the management server <NUM> is updated, a copy of the updated party management data is transmitted to the voice chat apparatus <NUM> used by the users participating in the party associated with the party management data. Then, the copy of the party management data stored in the storage units 20b of the voice chat apparatus <NUM> is updated. In this way, in the present embodiment, the latest information described in the party management data is shared between the voice chat apparatus <NUM> used by the users participating in the party associated with the party management data.

The following description assumes that the five users described in the party management data of <FIG> have voice chat.

<FIG> is a diagram illustrating exemplary processing that is executed when the user A inputs voice for voice chat in a case where the party management data is as illustrated in <FIG>. In this case, in each of the voice chat systems <NUM>-<NUM> to <NUM>-<NUM>, the voice chat apparatus <NUM> included in the corresponding voice chat system <NUM> executes a party process <NUM>.

When the user A inputs voice through the microphone 20f of the voice chat apparatus <NUM>-<NUM>, voice data indicating the voice is input to the party process <NUM> of the voice chat apparatus <NUM>-<NUM> (see (<NUM>) in <FIG>). Then, the party process <NUM> transmits the input voice data to the party process <NUM> of the voice chat apparatus <NUM> used by the other users participating in the same party as the user A (see (<NUM>) in <FIG>). Here, for example, voice data associated with the user ID of the user A may be transmitted. In <FIG>, the voice chat apparatus <NUM>-<NUM> is illustrated as an exemplary transmission destination of the voice data, but similar voice data is transmitted to the voice chat apparatuses <NUM>-<NUM> to <NUM>-<NUM>. Then, the party process <NUM> that has received the voice data outputs the voice indicated by the voice data from the speaker <NUM> (see (<NUM>) in <FIG>).

In a similar manner, voice input by each of the user B to the user E is output from the voice chat apparatus <NUM> used by the other users participating in the same party as the user.

<FIG> is a diagram illustrating other exemplary party management data. The party management data exemplified in <FIG> indicates that the voice chat assistance service is available in the voice chat systems <NUM>-<NUM> and <NUM>-<NUM>, but is unavailable in the voice chat systems <NUM>-<NUM> to <NUM>-<NUM>.

Note that, the following description assumes that the voice chat system <NUM>-<NUM> includes the voice chat apparatus <NUM>-<NUM> and an auxiliary apparatus <NUM>-<NUM>, and that the voice chat system <NUM>-<NUM> includes the voice chat apparatus <NUM>-<NUM> and an auxiliary apparatus <NUM>-<NUM>.

In the present embodiment, for example, when the user A performs predetermined operation to enable the voice chat assistance service in the voice chat system <NUM>-<NUM>, the party management data stored in the management server <NUM> is updated. Here, for example, the value of the assistance service use flag of the user data having the user ID of aaa is updated from <NUM> to <NUM>. Then, in response to this, the party management data stored in the voice chat apparatuses <NUM>-<NUM> to <NUM>-<NUM> is also updated.

Further, in a similar manner, when the user B performs predetermined operation to enable the voice chat assistance service in the voice chat system <NUM>-<NUM>, the party management data stored in the management server <NUM> and the voice chat apparatuses <NUM>-<NUM> to <NUM>-<NUM> is updated. Here, for example, the value of the assistance service use flag of the user data having the user ID of bbb is updated from <NUM> to <NUM>.

<FIG> is a diagram illustrating exemplary processing that is executed when the user A inputs text to be converted into voice for voice chat in a case where the party management data is as illustrated in <FIG>. Also in this example, in each of the voice chat systems <NUM>-<NUM> to <NUM>-<NUM>, the voice chat apparatus <NUM> included in the corresponding voice chat system <NUM> executes the party process <NUM>.

Further, the auxiliary apparatus <NUM> of the voice chat system <NUM> in which the voice chat assistance service is available executes a companion application process <NUM>. Then, the voice chat apparatus <NUM> of the voice chat system <NUM> executes a proxy process <NUM> for communication with the companion application process <NUM>. Here, for example, the auxiliary apparatuses <NUM>-<NUM> and <NUM>-<NUM> execute the companion application process <NUM>. Then, the voice chat apparatus <NUM>-<NUM> executes the proxy process <NUM> for communication with the companion application process <NUM> of the auxiliary apparatus <NUM>-<NUM>. Further, the voice chat apparatus <NUM>-<NUM> executes the proxy process <NUM> for communication with the companion application process <NUM> of the auxiliary apparatus <NUM>-<NUM>.

Further, in the present embodiment, as illustrated in <FIG>, the voice chat assistance service is available in at least one of the voice chat systems <NUM> used by the users participating in the party. In this case, all the voice chat apparatus <NUM> used by the users participating in the party execute the voice agent process <NUM>.

For example, when the value of the assistance service use flag of any of the user data included in the party management data stored in the voice chat apparatus <NUM> is updated to <NUM>, the voice chat apparatus <NUM> starts the voice agent process <NUM>.

Here, for example, the voice chat apparatuses <NUM>-<NUM> to <NUM>-<NUM> execute the voice agent process <NUM>. Note that, in the present embodiment, although the voice chat assistance service is unavailable in the voice chat systems <NUM>-<NUM> to <NUM>-<NUM>, the voice chat apparatuses <NUM>-<NUM> to <NUM>-<NUM> execute the voice agent process <NUM>.

Here, for example, the user A inputs text to the touch panel 24d of the auxiliary apparatus <NUM>-<NUM> (see (<NUM>) in <FIG>). Then, the companion application process <NUM> of the auxiliary apparatus <NUM>-<NUM> transmits text data including the text to the proxy process <NUM> of the voice chat apparatus <NUM>-<NUM> (see (<NUM>) in <FIG>). Then, in the voice chat apparatus <NUM>-<NUM>, the proxy process <NUM> outputs the text data to the voice agent process <NUM> and the party process <NUM> (see (<NUM>) and (<NUM>) in <FIG>).

Then, the voice agent process <NUM> of the voice chat apparatus <NUM>-<NUM> transmits the text data to the voice agent server <NUM> (see (<NUM>) in <FIG>). Then, the voice agent server <NUM> executes voice synthesis processing on the text data, and transmits voice data obtained as a result of the processing to the voice agent process <NUM> of the voice chat apparatus <NUM>-<NUM> (see (<NUM>) in <FIG>). Then, the voice agent process <NUM> outputs the voice data to the party process <NUM> (see (<NUM>) in <FIG>).

Then, the party process <NUM> of the voice chat apparatus <NUM>-<NUM> identifies the other voice chat systems <NUM> in which the voice chat assistance service is available. Here, for example, the voice chat system <NUM>-<NUM> is identified. Then, the party process <NUM> of the voice chat apparatus <NUM>-<NUM> transmits the voice data and text data described above to the party process <NUM> of the voice chat apparatus <NUM> included in the identified voice chat system <NUM> (see (<NUM>) in <FIG>). Here, for example, voice data and text data associated with the user ID of the user A may be transmitted.

Then, the party process <NUM> of the voice chat apparatus <NUM>-<NUM> outputs the received text data to the proxy process <NUM> (see (<NUM>) in <FIG>). Then, the proxy process <NUM> of the voice chat apparatus <NUM>-<NUM> transmits the text data to the companion application process <NUM> of the auxiliary apparatus <NUM>-<NUM> (see (<NUM>) in <FIG>). Then, the companion application process <NUM> of the auxiliary apparatus <NUM>-<NUM> displays the text included in the text data on the touch panel 24d (see (<NUM>) in <FIG>). Further, the party process <NUM> of the voice chat apparatus <NUM>-<NUM> may output the voice indicated by the received voice data from the speaker <NUM> (see (<NUM>) in <FIG>).

Further, the party process <NUM> of the voice chat apparatus <NUM>-<NUM> identifies the other voice chat systems <NUM> in which the voice chat assistance service is unavailable. Here, for example, the voice chat systems <NUM>-<NUM> to <NUM>-<NUM> are identified. Then, the party process <NUM> of the voice chat apparatus <NUM>-<NUM> transmits only the voice data described above to the party process <NUM> of the voice chat apparatus <NUM> included in the identified voice chat systems <NUM> (see (<NUM>) in <FIG>). Here, for example, voice data associated with the user ID of the user A may be transmitted. The text data described above is not transmitted to the party process <NUM> of the voice chat systems <NUM> in which the voice chat assistance service is unavailable. In <FIG>, the voice data is transmitted to the party process <NUM> of the voice chat apparatus <NUM>-<NUM> that is a representative. Then, the party process <NUM> of the voice chat apparatus <NUM>-<NUM> outputs the voice indicated by the received voice data from the speaker <NUM> (see (<NUM>) in <FIG>). Note that, in the present embodiment, in a similar manner, the voice indicated by the voice data described above is output from the speakers <NUM> of the voice chat apparatuses <NUM>-<NUM> and <NUM>-<NUM>.

<FIG> is a diagram illustrating exemplary processing that is executed when the user C inputs voice for voice chat in a case where the party management data is as illustrated in <FIG>.

When the user C inputs voice through the microphone 20f of the voice chat apparatus <NUM>-<NUM>, voice data indicating the voice is input to the party process <NUM> and the voice agent process <NUM> of the voice chat apparatus <NUM>-<NUM> (see (<NUM>) in <FIG>).

Then, the voice agent process <NUM> of the voice chat apparatus <NUM>-<NUM> transmits the voice data to the voice agent server <NUM> (see (<NUM>) in <FIG>). Then, the voice agent server <NUM> executes voice recognition processing on the voice data, and transmits text data obtained as a result of the processing to the voice agent process <NUM> (see (<NUM>) in <FIG>). Then, the voice agent process <NUM> outputs the text data to the party process <NUM> (see (<NUM>) in <FIG>).

Then, the party process <NUM> of the voice chat apparatus <NUM>-<NUM> identifies the other voice chat systems <NUM> in which the voice chat assistance service is available. Here, for example, the voice chat systems <NUM>-<NUM> and <NUM>-<NUM> are identified. Then, the party process <NUM> of the voice chat apparatus <NUM>-<NUM> transmits the voice data and text data described above to the party process <NUM> of the voice chat apparatus <NUM> included in the identified voice chat systems <NUM> (see (<NUM>) in <FIG>). Here, for example, voice data and text data associated with the user ID of the user C may be transmitted. In <FIG>, the voice data and the text data are transmitted to the party process <NUM> of the voice chat apparatus <NUM>-<NUM> that is a representative.

Then, the party process <NUM> of the voice chat apparatus <NUM>-<NUM> outputs the received text data to the proxy process <NUM> (see (<NUM>) in <FIG>). Then, the proxy process <NUM> of the voice chat apparatus <NUM>-<NUM> transmits the text data to the companion application process <NUM> of the auxiliary apparatus <NUM>-<NUM> (see (<NUM>) in <FIG>). Then, the companion application process <NUM> of the auxiliary apparatus <NUM>-<NUM> displays the text included in the text data on the touch panel 24d (see (<NUM>) in <FIG>). Further, the party process <NUM> of the voice chat apparatus <NUM>-<NUM> outputs the voice indicated by the received voice data from the speaker <NUM> (see (<NUM>) in <FIG>). Note that, in the present embodiment, in a similar manner, the auxiliary apparatus <NUM>-<NUM> displays the text included in the text data on the touch panel 24d. Here, the voice indicated by the voice data described above is output from the speaker <NUM> of the voice chat apparatus <NUM>-<NUM>.

Further, the party process <NUM> of the voice chat apparatus <NUM>-<NUM> identifies the other voice chat systems <NUM> in which the voice chat assistance service is unavailable. Here, for example, the voice chat systems <NUM>-<NUM> and <NUM>-<NUM> are identified. Then, the party process <NUM> of the voice chat apparatus <NUM>-<NUM> transmits only the voice data described above to the party process <NUM> of the voice chat apparatus <NUM> included in the identified voice chat systems <NUM> (see (<NUM>) in <FIG>). Here, for example, voice data associated with the user ID of the user C may be transmitted. The text data described above is not transmitted to the party process <NUM> of the voice chat systems <NUM> in which the voice chat assistance service is unavailable. In <FIG>, the voice data is transmitted to the party process <NUM> of the voice chat apparatus <NUM>-<NUM> that is a representative. Then, the party process <NUM> of the voice chat apparatus <NUM>-<NUM> outputs the voice indicated by the received voice data from the speaker <NUM> (see (<NUM>) in <FIG>). Note that, in the present embodiment, in a similar manner, the voice indicated by the voice data described above is output from the speaker <NUM> of the voice chat apparatus <NUM>-<NUM>.

<FIG> is a diagram illustrating an exemplary auxiliary screen that is displayed on the touch panel 24d of the auxiliary apparatus <NUM> according to the present embodiment. <FIG> illustrates the auxiliary screen that is displayed on the touch panel 24d of the auxiliary apparatus <NUM>-<NUM> in which the voice chat assistance service is available, which is used by the user A.

On the auxiliary screen illustrated in <FIG>, text obtained as a result of voice recognition on voice input by the users other than the user A is displayed. For example, on the auxiliary screen illustrated in <FIG>, text S1 obtained as a result of voice recognition on voice input by the user B is displayed in association with a character string S2 representing the user ID of the user B. The user ID of the user B can be identified on the basis of voice data and text data transmitted from the voice chat apparatus <NUM>-<NUM> in association with the user ID of the user B, for example.

Further, on the auxiliary screen illustrated in <FIG>, text S3 obtained as a result of voice recognition on voice input by the user C is displayed in association with a character string S4 representing the user ID of the user C. The user ID of the user C can be identified on the basis of voice data and text data transmitted from the voice chat apparatus <NUM>-<NUM> in association with the user ID of the user C, for example.

Further, on the auxiliary screen, a form F for text input and a send button SB for posting text input in the form F are displayed. For example, the user A inputs text in the form F and taps the send button SB to transmit text data including the text to the voice chat apparatus <NUM>-<NUM>. Further, on the auxiliary screen, text S5 input by the user A in this way is displayed in association with a character string S6 representing the user ID of the user A.

In a case where, in the present embodiment, the voice chat assistance service is available in none of the voice chat systems <NUM> used by the users participating in the party, the voice agent process <NUM> ends in all the voice chat apparatus <NUM> used by the users participating in the party.

For example, when the values of the assistance service use flags of all the user data included in the party management data stored in the voice chat apparatus <NUM> are set to <NUM>, the voice chat apparatus <NUM> ends the voice agent process <NUM>.

As described above, in the present embodiment, text obtained as a result of voice recognition on voice in voice chat is not provided to the voice chat systems <NUM> in which the voice chat assistance service is unavailable. Thus, according to the present embodiment, as compared to a case where text obtained as a result of voice recognition on voice in voice chat is provided to all the voice chat systems <NUM>, the data traffic for voice chat is reduced. In this way, according to the present embodiment, whether or not to provide text obtained as a result of voice recognition on voice in voice chat can be appropriately controlled.

Now, functions that are implemented in the voice chat system <NUM> according to the present embodiment are described in more detail.

<FIG> is a functional block diagram illustrating exemplary functions that are implemented in the voice chat system <NUM> according to the present embodiment. Note that, in the voice chat system <NUM> according to the present embodiment, all the functions illustrated in <FIG> are not necessarily implemented, and functions other than the functions illustrated in <FIG> may be implemented.

As illustrated in <FIG>, the voice chat apparatus <NUM> according to the present embodiment functionally includes, for example, a party management data storing unit <NUM>, a party managing unit <NUM>, a voice receiving unit <NUM>, a text acquiring unit <NUM>, a text receiving unit <NUM>, a voice acquiring unit <NUM>, a transmission control unit <NUM>, a data receiving unit <NUM>, a voice output unit <NUM>, and an auxiliary transmission unit <NUM>.

The party management data storing unit <NUM> is implemented mainly with the storage unit 20b. The party managing unit <NUM> and the transmission control unit <NUM> are implemented mainly with the processor 20a and the communication unit 20c. The voice receiving unit <NUM> is implemented mainly with the microphone 20f and the encoding/decoding unit <NUM>. The text acquiring unit <NUM>, the text receiving unit <NUM>, the voice acquiring unit <NUM>, the data receiving unit <NUM>, and the auxiliary transmission unit <NUM> are implemented mainly with the communication unit 20c. The voice output unit <NUM> is implemented mainly with the speaker <NUM> and the encoding/decoding unit <NUM>.

The above-mentioned functions are implemented by the processor 20a executing a program including instructions corresponding to the above-mentioned functions, which has been installed on the voice chat apparatus <NUM> that is the computer. The program is supplied to the voice chat apparatus <NUM> through a computer readable information storage medium such as an optical disc, a magnetic disk, a magnetic tape, a magneto-optical disk, or a flash memory, or via the Internet, for example.

Further, as illustrated in <FIG>, the auxiliary apparatus <NUM> according to the present embodiment functionally includes, for example, a text receiving unit <NUM>, a text transmitting unit <NUM>, an auxiliary reception unit <NUM>, and a display control unit <NUM>. The text receiving unit <NUM> and the display control unit <NUM> are implemented mainly with the processor 24a and the touch panel 24d. The text transmitting unit <NUM> and the auxiliary reception unit <NUM> are implemented mainly with the communication unit 24c.

The above-mentioned functions are implemented by the processor 24a executing a program including instructions corresponding to the above-mentioned functions, which has been installed on the auxiliary apparatus <NUM> that is the computer. The program is supplied to the auxiliary apparatus <NUM> through a computer readable information storage medium such as an optical disc, a magnetic disk, a magnetic tape, a magneto-optical disk, or a flash memory, or via the Internet, for example.

The party management data storing unit <NUM> of the present embodiment stores, for example, the party management data exemplified in <FIG> and <FIG>.

The party managing unit <NUM> of the present embodiment updates, for example, when receiving party management data transmitted from the management server <NUM>, the party management data stored in the party management data storing unit <NUM> to the received party management data.

In the present embodiment, the value of the assistance service use flag in the party management data stored in the management server <NUM> is updated when the user performs operation to start using the voice chat assistance service or operation to stop using the voice chat assistance service, for example. Then, the management server <NUM> transmits, on the basis of the update, the updated party management data to the voice chat system <NUM> used by the user participating in the party managed by the party management data. Then, as described above, the party managing unit <NUM> updates, when receiving the party management data transmitted from the management server <NUM>, the party management data stored in the party management data storing unit <NUM> to the received party management data.

Further, the party managing unit <NUM> may detect, on the basis of the updated party management data, that the display of voice recognition results is enabled in any of the voice chat systems <NUM>. The detection includes, for example, detecting that at least one of the values of the assistance service use flags that have been all <NUM> is changed to <NUM>.

Further, the party managing unit <NUM> may detect, on the basis of the updated party management data, that the display of voice recognition results is disabled in all the voice chat systems <NUM>. The detection includes, for example, detecting that at least one of the values of the assistance service use flags that has been <NUM> is changed so that the values of all the assistance service use flags are <NUM>.

The voice receiving unit <NUM> of the present embodiment receives, for example, voice in voice chat. The voice receiving unit <NUM> may encode the voice to generate voice data indicating the voice.

The text acquiring unit <NUM> of the present embodiment acquires, for example, text obtained as a result of voice recognition on voice received by the voice receiving unit <NUM>. Here, for example, the text acquiring unit <NUM> may transmit voice data indicating the voice to the voice agent server <NUM> capable of communicating with the voice chat apparatus <NUM>. Then, the text acquiring unit <NUM> may receive, from the voice agent server <NUM>, text data including text obtained as a result of voice recognition on the voice indicated by the voice data. This function corresponds to the function of the voice agent process <NUM> illustrated in <FIG> and <FIG>.

Further, the text acquiring unit <NUM> may start acquiring text when the display of voice recognition results is enabled in any of the at least one voice chat system <NUM>. Further, the text acquiring unit <NUM> may start acquiring text when the auxiliary apparatus <NUM> is included in any of the at least one voice chat system <NUM>. For example, the text acquiring unit <NUM> may start the voice agent process <NUM> when the party managing unit <NUM> detects that the display of text is enabled in any of the voice chat systems <NUM>.

Further, the text acquiring unit <NUM> may stop acquiring text when the display of voice recognition results is disenabled in all of the at least one voice chat system <NUM>. Further, the text acquiring unit <NUM> may stop acquiring text when the auxiliary apparatus <NUM> is included in none of the at least one voice chat system <NUM>. For example, the text acquiring unit <NUM> may end the voice agent process <NUM> when the party managing unit <NUM> detects that the display of text is disenabled in all the voice chat systems <NUM>.

The text receiving unit <NUM> of the voice chat apparatus <NUM> of the present embodiment receives, for example, text to be subjected to voice synthesis processing. Here, the text receiving unit <NUM> may receive, for example, text input to the auxiliary apparatus <NUM> connected to the voice chat apparatus <NUM>. This function corresponds to the function of the proxy process <NUM> illustrated in <FIG> and <FIG>.

The voice acquiring unit <NUM> of the present embodiment acquires, for example, voice obtained as a result of voice synthesis on text received by the text receiving unit <NUM>. Here, for example, the voice acquiring unit <NUM> may transmit text data indicating the text to the voice agent server <NUM> capable of communicating with the voice chat apparatus <NUM>. Then, the voice acquiring unit <NUM> may receive, from the voice agent server <NUM>, voice obtained as a result of voice synthesis on the text included in the text data. This function corresponds to the function of the voice agent process <NUM> illustrated in <FIG> and <FIG>.

The transmission control unit <NUM> of the present embodiment controls, for example, on the basis of whether or not the display of voice recognition results is enabled in the voice chat system <NUM> that is a communication destination, whether or not to transmit text data to the communication destination. Here, the transmission control unit <NUM> may control, on the basis of whether or not the display of voice recognition results is enabled in the voice chat system <NUM> that is a communication destination, whether or not to transmit, to the communication destination, voice data or the voice data and text data. For example, the transmission control unit <NUM> may control whether not to transmit voice data indicating voice received by the voice receiving unit <NUM> or to transmit, together with the voice data, text data acquired by the text acquiring unit <NUM>. Further, for example, the transmission control unit <NUM> may control whether or not to transmit voice data indicating voice acquired by the voice acquiring unit <NUM> or to transmit, together with the voice data, text data including text received by the text receiving unit <NUM>. This function corresponds to the function of the party process <NUM> illustrated in <FIG>.

Here, the transmission control unit <NUM> may control, for example, on the basis of whether or not the voice chat system <NUM> that is a communication destination includes the auxiliary apparatus <NUM> configured to display voice recognition results, whether or not to transmit text data to the communication destination. Further, the transmission control unit <NUM> may control, for example, on the basis of whether or not the voice chat system <NUM> that is a communication destination includes the auxiliary apparatus <NUM> configured to display voice recognition results, whether or not to transmit voice data or the voice data and text data.

Further, for example, on the basis of the values of the assistance service use flags in the party management data stored in the party management data storing unit <NUM>, whether or not to transmit text data to a communication destination may be controlled. For example, voice data and text data may be transmitted to the voice chat system <NUM> having an assistance service use flag with a value of <NUM>. Meanwhile, only voice data may be transmitted to the voice chat system <NUM> having an assistance service use flag with a value of <NUM>.

The data receiving unit <NUM> of the present embodiment receives, for example, voice data transmitted from the voice chat apparatus <NUM> that is a communication destination. Further, the data receiving unit <NUM> of the present embodiment receives, for example, text data transmitted from the voice chat apparatus <NUM> that is a communication destination. This function corresponds to the function of the party process <NUM> illustrated in <FIG>.

The voice output unit <NUM> of the present embodiment outputs, for example, voice in voice chat. For example, the voice output unit <NUM> outputs the voice indicated by voice data received by the data receiving unit <NUM>. The voice output unit <NUM> may decode voice data received by the data receiving unit <NUM> to output the voice indicated by the voice data.

The auxiliary transmission unit <NUM> of the present embodiment transmits, for example, text data received by the data receiving unit <NUM> to the auxiliary apparatus <NUM>. This function corresponds to the function of the proxy process <NUM> illustrated in <FIG> and <FIG>.

The text receiving unit <NUM> of the auxiliary apparatus <NUM> of the present embodiment receives, for example, text input to the touch panel 24d.

The text transmitting unit <NUM> of the present embodiment transmits, for example, text data including text received by the text receiving unit <NUM> to the voice chat apparatus <NUM>.

The auxiliary reception unit <NUM> of the present embodiment receives, for example, text data transmitted from the voice chat apparatus <NUM>.

The display control unit <NUM> of the present embodiment displays, for example, the text included in text data received by the auxiliary reception unit <NUM> or text received by the text receiving unit <NUM> on the touch panel 24d, for example. The display control unit <NUM> may display the auxiliary screen illustrated in <FIG> on the touch panel 24d.

The functions of the text receiving unit <NUM>, the text transmitting unit <NUM>, the auxiliary reception unit <NUM>, and the display control unit <NUM> of the auxiliary apparatus <NUM> correspond to the function of the companion application process <NUM> illustrated in <FIG> and <FIG>.

Here, an exemplary flow of processing that is performed in the voice chat apparatus <NUM> in which the voice agent process <NUM> has stopped according to the present embodiment is described with reference to the flow chart of <FIG>. The processing in S101 to S102 illustrated in <FIG> is repeatedly executed at a predetermined sampling rate.

First, the voice receiving unit <NUM> encodes voice received in the period of this loop to generate voice data (S101).

Then, the transmission control unit <NUM> transmits the voice data generated in the processing in S101 to the voice chat apparatus <NUM> used by a user participating in the same party (S102), and the processing returns to the processing in S101. Note that, the voice data is not transmitted to the voice chat apparatus <NUM> that executes the processing in S102.

The voice chat apparatus <NUM> that has received the voice data transmitted in the processing in S102 outputs the voice indicated by the voice data.

Next, an exemplary flow of processing based on input voice that is performed in the voice chat apparatus <NUM> in which the voice agent process <NUM> has been operating according to the present embodiment is described with reference to the flow chart of <FIG>. The processing in S201 to S207 illustrated in <FIG> is repeatedly executed at a predetermined sampling rate.

First, the voice receiving unit <NUM> encodes voice received in the period of this loop to generate voice data (S201).

Then, the text acquiring unit <NUM> transmits the voice data generated in the processing in S201 to the voice agent server <NUM> (S202).

Then, the text acquiring unit <NUM> receives the text data transmitted from the voice agent server <NUM> (S203).

Then, the transmission control unit <NUM> identifies, on the basis of the party management data stored in the party management data storing unit <NUM>, the voice chat apparatus <NUM> associated with user data having an assistance service use flag with a value of <NUM> (S204).

Then, the transmission control unit <NUM> transmits, to the voice chat apparatus <NUM> identified in the processing in S204, the voice data generated in the processing in S201 and the text data received in the processing in S203 (S205). Note that, the voice data and the text data are not transmitted to the voice chat apparatus <NUM> that executes the processing in S205.

Then, the transmission control unit <NUM> identifies, on the basis of the party management data stored in the party management data storing unit <NUM>, the voice chat apparatus <NUM> associated with user data having an assistance service use flag with a value of <NUM> (S206).

Then, the transmission control unit <NUM> transmits the voice data generated in the processing in S201 to the voice chat apparatus <NUM> identified in the processing in S206 (S207), and the processing returns to the processing in S201. Note that, the voice data is not transmitted to the voice chat apparatus <NUM> that executes the processing in S207.

The voice chat apparatus <NUM> that has received the voice data transmitted in the processing in S205 or S207 outputs the voice indicated by the voice data.

The voice chat apparatus <NUM> that has received the text data transmitted in the processing in S205 transmits the text data to the auxiliary apparatus <NUM> connected to the voice chat apparatus <NUM>. Then, the auxiliary apparatus <NUM> that has received the text data displays the text included in the text data on the touch panel 24d of the auxiliary apparatus <NUM>.

Note that, in the processing in S205, only the text data received in the processing in S203 may be transmitted. In this case, the voice chat apparatus <NUM> that has received the text data may not output the voice indicated by the voice data generated in the processing in S201.

Next, an exemplary flow of processing based on input text that is performed in the voice chat apparatus <NUM> in which the voice agent process <NUM> has been operating according to the present embodiment is described with reference to the flow chart of <FIG>. The processing in S301 to S307 illustrated in <FIG> is repeatedly executed at a predetermined sampling rate.

First, the text receiving unit <NUM> receives text data transmitted from the auxiliary apparatus <NUM> in the period of this loop (S301).

Then, the voice acquiring unit <NUM> transmits the text data generated in the processing in S301 to the voice agent server <NUM> (S302).

Then, the voice acquiring unit <NUM> receives the voice data transmitted from the voice agent server <NUM> (S303).

Then, the transmission control unit <NUM> identifies, on the basis of the party management data stored in the party management data storing unit <NUM>, the voice chat apparatus <NUM> associated with user data having an assistance service use flag with a value of <NUM> (S304).

Then, the transmission control unit <NUM> transmits, to the voice chat apparatus <NUM> identified in the processing in S304, the voice data received in the processing in S303 and the text data received in the processing in S301 (S305). Note that, the voice data and the text data are not transmitted to the voice chat apparatus <NUM> that executes the processing in S305.

Then, the transmission control unit <NUM> identifies, on the basis of the party management data stored in the party management data storing unit <NUM>, the voice chat apparatus <NUM> associated with user data having an assistance service use flag with a value of <NUM> (S306).

Then, the transmission control unit <NUM> transmits the voice data received in the processing in S303 to the voice chat apparatus <NUM> identified in the processing in S306 (S307), and the processing returns to the processing in S301. Note that, the voice data is not transmitted to the voice chat apparatus <NUM> that executes the processing in S307.

The voice chat apparatus <NUM> that has received the voice data transmitted in the processing in S305 or S307 outputs the voice indicated by the voice data.

The voice chat apparatus <NUM> that has received the text data transmitted in the processing in S305 transmits the text data to the auxiliary apparatus <NUM> connected to the voice chat apparatus <NUM>. Then, the auxiliary apparatus <NUM> that has received the text data displays the text included in the text data on the touch panel 24d of the auxiliary apparatus <NUM>.

Note that, in the processing in S305, only the text data received in the processing in S301 may be transmitted. In this case, the voice chat apparatus <NUM> that has received the text data may not output the voice indicated by the voice data generated in the processing in S303.

Note that, the present invention is not limited to the embodiment described above.

For example, the division of roles of the voice chat apparatus <NUM> and the auxiliary apparatus <NUM> is not limited to the above-mentioned one. For example, the auxiliary apparatus <NUM> may implement some or all of the functions of the voice chat apparatus <NUM> illustrated in <FIG>. Further, for example, the voice chat apparatus <NUM> may implement some or all of the functions of the auxiliary apparatus <NUM> illustrated in <FIG>.

Claim 1:
A communication system, the communication system comprising:
a first voice chat system (<NUM>-<NUM>) comprising:
a voice receiving unit (<NUM>) configured to receive voice in voice chat;
a text acquiring unit (<NUM>) configured to acquire text obtained as a result of voice recognition on the voice;
a transmission control unit (<NUM>) configured to control, on a basis of whether or not display of a voice recognition result is performed by an auxiliary apparatus in the voice chat system that is a communication destination, whether or not to transmit text data including the text to the communication destination; and
a transmission unit (<NUM>) configured to transmit the voice data and, in response to control by the transmission control unit, text data; and
a second voice chat system (<NUM>-<NUM>) comprising:
a voice chat apparatus (<NUM>-<NUM>) comprising a communication unit (<NUM>) configured to receive the voice and text data from the first voice chat system and configured to output the received voice via a speaker (<NUM>), and comprising a second communication unit (<NUM>) to transmit the text data to the auxiliary apparatus; and
the auxiliary apparatus (<NUM>-<NUM>) configured to output the received text data for display, wherein the auxiliary apparatus is a portable device executing a companion application process (<NUM>).