Terminal apparatus, information processing apparatus, and microphone voice transmission method

A transmission processing section 206 transmits information regarding an operation performed by a user to an information processing apparatus 10. An image acquisition section 208 acquires image data of a game that reflects the operation information from the information processing apparatus 10. A voice acquisition section 210 acquires voice data of the game. A reproduction processing section 212 reproduces the acquired game image data and game voice data. A microphone voice control section 202 controls whether or not to transmit a voice signal of a microphone 204 to the information processing apparatus 10. A display processing section 214 displays a microphone icon indicating status of the microphone 204.

TECHNICAL FIELD

The present invention relates to techniques for executing a game by use of a remotely located information processing apparatus.

BACKGROUND ART

Recent years have seen the widespread use of services that allow users to share game images coming from the ongoing game or captured of games via a sharing site. It has also become possible for users to play the game together by having their terminals connected with one another on a peer-to-peer basis (P2P). PTL 1 discloses two modes of game play: “multi-play” in which multiple users having the same game play it together online, and “remote play” in which game operation information is transmitted to a remotely located information processing apparatus that in turn returns game images having the operation information reflected in the game progress.

CITATION LIST

Patent Literature

SUMMARY

Technical Problem

When multiple players play games together, they can expand the scope of game play by communicating with each other via voice using voice applications such as voice chat. Thus it has been desired to build a scheme for sending the user's voice to each other during game play.

An object of the present invention is to provide techniques for transmitting a voice signal input to a microphone of a terminal apparatus to an information processing apparatus.

Solution to Problem

In solving the above problem and according to one embodiment of the present invention, there is provided a terminal apparatus including: a transmission processing section configured to transmit information regarding an operation performed by a user to an information processing apparatus; an acquisition section configured to acquire image data and voice data of a game that reflects the operation information from the information processing apparatus; a reproduction processing section configured to reproduce the acquired game image data and game voice data; and a microphone voice control section configured to control whether or not to transmit a voice signal of a microphone to the information processing apparatus.

According to another embodiment of the present invention, there is provided an information processing apparatus. The information processing apparatus includes: an execution section configured to generate image data and voice data of a game that reflects information regarding an operation performed by a user, the operation information being transmitted from a terminal apparatus; a transmission processing section configured to transmit the game image data and the game voice data to the terminal apparatus; a microphone voice acquisition section; and a microphone resource allocation section configured to allocate the microphone voice acquisition section to a microphone of the terminal apparatus.

According to a further embodiment of the present invention, there is provided a microphone voice transmission method. This method includes the steps of: transmitting information regarding an operation performed by a user to an information processing apparatus that executes a game; acquiring image data and voice data of the game that reflects the operation information from the information processing apparatus; reproducing the acquired game image data and game voice data; and controlling whether or not to transmit a voice signal of a microphone to the information processing apparatus.

Incidentally, if other combinations of the above-outlined composing elements or the above expressions of the present invention are converted between different forms such as a method, a device, a system, a recording medium, and a computer program, they still constitute effective embodiments of this invention.

DESCRIPTION OF EMBODIMENTS

FIG. 1depicts a basic mode of use of an information processing apparatus10embodying the present invention. The information processing apparatus10is connected by wire or wirelessly with an input device6operated by a user A. The input device6supplies the information processing apparatus10with operation information indicating input operations performed by the user A. Upon receipt of the operation information from the input device6, the information processing apparatus10has the received information reflected in the processing of system software and application software and causes an output device4to output the result of the processing. The information processing apparatus10of this embodiment is a game device that executes games. The input device6may be a game controller, for example, which supplies the information processing apparatus10with the operation information from the user A.

An auxiliary storage device2is a mass storage device such as a hard disk drive (HDD) or a flash memory. This device may be either an external storage device connected with the information processing apparatus10typically via a universal serial bus (USB) or an internal storage device. The output device4may be a TV set having a display for outputting images and speakers for outputting sounds. The output device4may be connected by wire or wirelessly with the information processing apparatus10.

An access point (called the AP hereunder)8has the functions of a wireless access point and a router. The information processing apparatus10connects by wire or wirelessly with the AP8so as to connect with an external network. A camera7is a stereo camera that captures images of the space around the output device4.

FIG. 2is a functional block diagram of the information processing apparatus10. The information processing apparatus10is configured to have a main power button20, a power-on light-emitting diode (LED)21, a standby LED22, a system controller24, a clock26, a device controller30, a media drive32, a USB module34, a flash memory36, a wireless communication module38, a wired communication module40, a subsystem50, and a main system60.

The main system60includes a main central processing unit (CPU), a memory and a controller constituting a main storage device, and a graphics processing unit (GPU). The GPU is mainly used in the arithmetic processing of game programs. These functions may be configured as a system-on-chip and formed in a single chip. The main CPU has the function of executing game programs recorded in the auxiliary storage device2or in a ROM medium44.

The subsystem50includes a sub CPU and a memory and a controller constituting a main storage device, but has neither the GPU nor the function of executing game programs. The number of circuit gates in the sub CPU is smaller than that in the main CPU. The operating power consumption of the sub CPU is lower than that of the main CPU. The sub CPU remains active while the main CPU is in a standby state. However, the sub CPU has limited processing functionality so as to minimize power consumption.

The main power button20is an input part on which the user performs input operations. Disposed at the front of an enclosure housing the information processing apparatus10, the main power button20is operated to turn on and off the supply of power to the main system60of the information processing apparatus10. The power-on LED21is lit when the main power button20is turned on. The standby LED22is lit when the main power button20is turned off.

The system controller24detects whether the main power button20is pressed by the user. With the main power supply turned off, pressing the main power button20causes the system controller24to acquire the pressing operation as an on-instruction. On the other hand, with the main power supply turned on, pressing the main power button20causes the system controller24to acquire the pressing operation as an off-instruction.

The clock26is a real-time clock that generates the current date and time information and feeds it to the system controller24, subsystem50, and main system60. The device controller30is configured as a large-scale integrated circuit (LSI) that mediates, like Southbridge, transfers of information between devices. As illustrated, the device controller30is connected with such devices as the system controller24, media drive32, USB module34, flash memory36, wireless communication module38, wired communication module40, subsystem50, and main system60. The device controller30controls data transfer timing by absorbing the differences in electrical characteristics and in data transfer rates between the devices.

The media drive32is a drive device that drives an attached ROM medium44in order to read such resources as programs and data therefrom, the ROM medium44having application software such as games as well as license information recorded therein. The ROM medium44may be any one of read-only recording media such as optical disks, magneto-optical disks, and Blu-ray disks.

The USB module34is connected with an external device by USB cable. The USB module34may be connected with the auxiliary storage device2and camera7by USB cable. The flash memory36is an auxiliary storage device that constitutes internal storage. The wireless communication module38communicates wirelessly, for example, with the input device6according to a communication protocol such as the Bluetooth (registered trademark) protocol or IEEE 802.11 protocol. The wired communication module40communicates by wire with an external device in order to connect with an external network via the AP8. The user A operating the information processing apparatus10may enjoy an online game with another user operating another information processing apparatus.

FIG. 3depicts one mode of use of an information processing apparatus embodying the present invention. In the information processing system1, a management server5provides the user with network services for games. The management server5manages a network account that identifies the user. Using the network account, the user signs in to a network service provided by the management server5.

FIG. 3illustrates how the information processing apparatus10operated by the user A is connected with an information processing apparatus10aoperated by a user B via a network3. The information processing apparatus10and the information processing apparatus10aexecute the same game, each apparatus transmitting to the other party the operation information regarding the input device6operated by the own user. The information processing apparatuses10and10acause the operation information from the users A and B to be reflected in their game progress for purpose of game progress synchronization. The users A and B can thus enjoy playing the game together.

At this point, the information processing apparatuses10and10aeach transmit to the other party the own user's voice input to the respective microphones (not depicted) together with the user's operation information. This allows the users A and B to have a conversation with each other while playing the game. The users can thus consult with each other typically about the game walkthrough in a coordinated play.

FIG. 4depicts another mode of use of the information processing apparatus embodying the present invention. The drawing illustrates how a terminal apparatus14operated by the user A connects with the information processing apparatus10via the network3to allow the user A to play the game. The mode in which the user A operates the information processing apparatus10from a remote location to play the game is called “remote play.” In remote play, the user A is also required to sign in to the management server5so as to let the information processing apparatus10communicate with the terminal apparatus14.

To perform remote play, the user A operates the terminal apparatus14to transmit a connection request to the information processing apparatus10. If the main power supply of the information processing apparatus10remains off at this point, the main system60is activated in accordance with the connection request. The main system60generates menu image data representing an array of game icons and transmits the generated data to the terminal apparatus14. In turn, the terminal apparatus14displays a menu screen on a display device. When the user A selects a desired game icon on the menu screen, the terminal apparatus14transmits the operation information regarding the selection to the information processing apparatus10. The information processing apparatus10starts the selected game, generates game image data, and transmits the generated data to the terminal apparatus14. In turn, the terminal apparatus14displays a game start screen on the display device.

In remote play, the information processing apparatus10accepts as input to the game the operation information input to the terminal apparatus14by the user A. Also, the information processing apparatus10transmits to the terminal apparatus14the output data being sent to the output device4. This enables the terminal apparatus14to display on the display device the game image reflecting the operations of the user A. The terminal apparatus14may be an information processing apparatus such as a portable game device, a smartphone, or a tablet. The terminal apparatus14may alternatively be a personal computer. The terminal apparatus14is connected with the information processing apparatus10on a peer-to-peer basis.

FIG. 5depicts still another mode of use of the information processing apparatus embodying the present invention. The drawing illustrates a mixture of the mode of use inFIG. 3and the mode of use inFIG. 4, picturing how the user A in a location away from the information processing apparatus10plays the game with the user B in remote play. In the mode of play depicted inFIG. 5, the user A operates the information processing apparatus10remotely from the terminal apparatus14so as to connect the information processing apparatus10with the information processing apparatus10avia the network3. The connection allows the users A and B to play the game together.

FIG. 6depicts configurations of the information processing apparatus10and the terminal apparatus14. The information processing apparatus10includes an information acquisition section100, a menu image generation section102, an execution section112, a microphone resource allocation section118, a microphone voice acquisition section120, and a transmission processing section122. The terminal apparatus14includes an acceptance section200, a microphone voice control section202, a microphone204, a transmission processing section206, an image acquisition section208, a voice acquisition section210, a reproduction processing section212, and a display processing section214.

The components noted inFIG. 6as the functional blocks for performing diverse processes may be configured by hardware in the form of circuit blocks, memories, and other LSI, for example, or by software using typically programs loaded into a memory. It will thus be appreciated by those skilled in the art that these functional blocks are configured by hardware only, by software only, or by a combination thereof in diverse forms and are not limited to any one of such forms.

The user A operates the terminal apparatus14to transmit to the information processing apparatus10a connection request for remote play. Once a connection is established between the terminal apparatus14and the information processing apparatus10for remote play, the menu image generation section102generates menu image data representing an array of game icons. The transmission processing section122transmits the menu image data to the terminal apparatus14.

FIG. 7depicts a typical menu screen displayed on the display device of the terminal apparatus14. The image acquisition section208acquires the menu image data transmitted from the information processing apparatus10. The reproduction processing section212displays the menu screen on the display device. The display device of the terminal apparatus14is configured as a touch screen. When the user A selects a desired game icon on the menu screen, the acceptance section200accepts information regarding the selection. The transmission processing section206transmits the selection information to the information processing apparatus10. When the information acquisition section100in the information processing apparatus10accepts the transmitted selection information, the execution section112executes the selected game. At this point, a game image generation section114generates game image data while a game voice generation section116generates game voice data. The transmission processing section122transmits the game image data and the game voice data to the terminal apparatus14.

In the terminal apparatus14, the acceptance section200accepts the operation information regarding the game played by the user A. For example, the user A may either connect the input device6with the terminal apparatus14to input the operation information, or operate a GUI imitating the input device6on the touch screen of the input terminal14for operation information input. The transmission processing section206transmits the operation information from the user A to the information processing apparatus10. In the information processing apparatus10, the execution section112causes the operation information to be reflected in the game progress. The game image generation section114and the game voice generation section116generate respectively the game image data and the game voice data reflecting the operation information. The transmission processing section122transmits the game image data and the game voice data to the terminal apparatus14.

In the terminal apparatus14, the image acquisition section208acquires the game image data reflecting the operation information while the voice acquisition section210acquires the game voice data reflecting the operation information. The reproduction processing section212reproduces the game image data through the display device and the game voice data through speakers. Thus in remote play, the user A in a location away from the information processing apparatus10may use the terminal apparatus14to operate the information processing apparatus10.

With the embodiment, especially in the mode of use depicted inFIG. 5, the user A in remote play may desire to convey his or her voice to the user B via the terminal apparatus14. To address the desire, the terminal apparatus14incorporates the function of transmitting the voice signal output from the microphone204to the information processing apparatus10. Specifically in the terminal apparatus14, the microphone voice control section202controls whether or not to transmit the voice signal from the microphone204to the information processing apparatus10in accordance with the operation of the user A. The microphone204may be either a built-in microphone or an external microphone attached to the terminal apparatus14.

The display processing section214displays a microphone icon indicating the status of the microphone204. On the menu screen depicted inFIG. 7, for example, the user A may tap the screen space other than the displayed icons. This causes the display processing section214to display a tool bar at the bottom of the screen. The display processing section214displays on the tool bar a microphone icon indicating the status of the microphone204. The microphone icon is displayed in such a manner that it can be tapped for operation by the user.

FIG. 8depicts a tool bar superimposed on the menu screen. A tool bar300indicates various icons including a microphone icon302. These icons are displayed in such a manner that each of them can be tapped for operation.FIG. 9depicts display examples of microphone icons.FIG. 9(a)depicts a microphone icon indicating a mute release state. The mute release state of the microphone204is a state in which the voice signal converted by the microphone204can be used.

FIG. 9(b)depicts a microphone icon indicating a mute state of the microphone204. The mute state of the microphone204is a state in which a voice conversion function of the microphone204is stopped or in which the voice signal converted by the microphone cannot be used.FIG. 9(c)depicts an animated microphone icon indicating that the microphone204is receiving input of at least a predetermined level of sound volume.FIG. 9(c)gives a special effect display of a microphone icon which indicates the mute release state and which is supplemented with an animation that varies with sound volume.

The user can verify the status of the microphone icon302by checking it on the tool bar300.

As depicted inFIG. 8, the microphone204is in a mute state immediately after the start of remote play. When the microphone icon indicating the mute state is tapped, the microphone voice control section202causes the transmission processing section206to transmit to the information processing apparatus10notification information indicating that the voice signal of the microphone204is to be transmitted to the information processing apparatus10. When the information acquisition section100in the information processing apparatus10acquires the notification information, the information acquisition section100forwards the acquired notification information to the microphone resource allocation section118. In turn, the microphone resource allocation section118performs the process of virtually allocating the microphone204of the terminal apparatus14to a microphone resource possessed by the information processing apparatus10.

The information processing apparatus10possesses multiple microphone voice acquisition sections120as its microphone resources that receive voice signals from microphones by wire or wirelessly. Upon receipt of the notification information, the microphone resource allocation section118allocates one microphone resource (microphone voice acquisition section120) to the microphone204. When allocated to the microphone204, the microphone voice acquisition section120is enabled to acquire the voice signal of the microphone204transmitted from the terminal apparatus14. Once the microphone resource allocation section118allocates the microphone voice acquisition section120to the microphone204, the allocation is not released until remote play is terminated. Thus after transmitting the notification information once to the information processing apparatus10via the transmission processing section206, the microphone voice control section202prevents the transmission processing section206from sending the notification information again to the information processing apparatus10from that point on even if the microphone icon indicating the mute state of the microphone204is again operated. That is, during remote play, the microphone voice control section202transmits the notification information only once to the information processing apparatus10.

When the microphone icon indicating the mute state of the microphone204(seeFIG. 9(b)) is tapped, the display processing section214displays the microphone icon indicating the mute release state of the microphone204(seeFIG. 9(a)). When the microphone indicating the mute release state of the microphone204(seeFIG. 9(a)) is tapped, the display processing section214displays the microphone icon indicating the mute state of the microphone204(seeFIG. 9(b)). In this manner, the display processing section214switches the mode of display of the microphone icon302in keeping with the user's operations on the microphone icon302.

The mode of display of the microphone icon302is switched by the display processing section214under voice signal transmission control of the microphone voice control section202. That is, when the microphone icon indicating the mute state of the microphone204(seeFIG. 9(b)) is tapped, the microphone voice control section202causes the transmission processing section206to transmit the voice signal of the microphone204to the information processing apparatus10. When the microphone indicating the mute release state of the microphone204(seeFIG. 9(a)) is tapped, the microphone voice control section202causes the transmission processing section206to stop transmitting the voice signal of the microphone204. In this manner, the microphone voice control section202controls whether or not to transmit the microphone voice signal in accordance with the user's operations on the microphone icon302.

For example, when the user A plays the game alone in the mode of use depicted inFIG. 4, there is little need to transmit the voice signal of the microphone204to the information processing apparatus10. However, when the user A plays the game together with the user B, it is preferred that the voice signal of the microphone204be transmitted to the information processing apparatus10that in turn will transfer the signal to the information processing apparatus10a. Thus the user A verifies the status of the microphone204by checking the mode of display of the microphone icon302and, as needed, taps the microphone icon302to determine whether or not to transmit the microphone voice signal.

In the information processing apparatus10, the microphone voice acquisition section120acquires the voice signal of the microphone204. In the case where the user A is in voice chat with the user B, the transmission processing section122transmits the voice signal of the microphone204to the information processing apparatus10aof the user B. Also, when the information acquisition section100acquires the microphone voice signal from the information processing apparatus10a, the transmission processing section122transmits the microphone voice signal of the user B to the terminal apparatus14. This allows the user A and user B to play the game together while having a conversation.

A voice quality management section124manages the voice quality of voice data transmitted between the information processing apparatus10and the terminal apparatus14. In the information processing system1, multiple kinds of voice data can be transmitted simultaneously between the information processing apparatus10and the terminal apparatus14. The voice quality management section124manages the voice quality of each kind of voice data by suitably setting a voice encoding format and a bit rate, for example.

An example of the voice data transmitted between the information processing apparatus10and the terminal apparatus14is game voice data that is generated by the game voice generation section116in remote play and transmitted by the transmission processing section122to the terminal apparatus14. A second example of the voice data is chat voice data transmitted between the information processing apparatus10and the terminal apparatus14when voice chat is conducted therebetween. A third example of the voice data is, as explained above in connection with the embodiment, microphone voice data that is input to the microphone204of the terminal apparatus14and transmitted to the information processing apparatus10via the transmission processing section206.

In the case where multiple kinds of voice data are transmitted simultaneously between the information processing apparatus10and the terminal apparatus14, the voice quality management section124sets the voice quality for each kind of voice data. Voice quality is determined by a combination of the voice encoding format and the bit rate, for example. Given the multiple kinds of voice data, the voice quality management section124may set the voice quality of the game voice data to a maximum level. The voice quality management section124may set the voice quality of the game voice data to the highest level while setting the voice quality of other voice data such as the chat voice data and the microphone voice data to lower levels. The voice quality management section124may alternatively set the voice quality level of the chat voice data and that of the microphone voice data in a manner adaptive to the communication environment.

The voice quality management section124may hold priorities of multiple kinds of voice data that can be transmitted between the information processing apparatus10and the terminal apparatus14in order to maximize voice quality. For example, in the case where the game voice data, chat voice data, and microphone voice data are prioritized in that order, the voice quality management section124may set the voice quality of the game voice data to the highest level followed by a lower voice quality level of the chat voice data and a still lower voice quality level the microphone voice data. For example, with the game voice data and the microphone voice data being transmitted between the information processing apparatus10and the terminal apparatus14, if a voice chat application is executed anew and the chat voice data starts to be transmitted, the voice quality management section124may change the voice quality of the microphone voice data to a lower level so as to let the voice quality of the microphone voice quality be set using desired voice parameters.

The present invention has been described above in conjunction with a specific embodiment. It is to be understood by those skilled in the art that suitable combinations of the constituent elements and various processes of the embodiment described above as an example will lead to further variations of the present invention and that such variations also fall within the scope of this invention. For example, whereas it was explained above in reference toFIG. 5that the user A performs remote play, the user B may alternatively perform remote play in place of the user A.

In another example, if the microphone204of the terminal apparatus14becomes unusable during remote play, the tool bar300may display a microphone icon302indicating that the microphone204is currently not usable.

REFERENCE SIGNS LIST

INDUSTRIAL APPLICABILITY

The present invention may be applied to the techniques for executing games.