Patent Description:
There has been technology for transmitting a video from a server to a terminal. Patent Literature <NUM> discloses an image processing system capable of playing a video through efficient priority control of video streams that are transmitted/received over a network by attaching, as a priority tag, to headers of packets of a video stream, an importance level which is set for each image frame, and then transmitting the packets with priority tags to a network that is constituted of routers having a packet priority control function.

[Patent Literature <NUM>] <CIT>. Other previously known arrangements are disclosed in <CIT>, <CIT>, <CIT>, <CIT>, and <CIT>.

An example of possible methods by which a client outputs and displays a screen for displaying a plurality of videos is to distribute by streaming a plurality of videos from a server to a client so that the client generates a screen by combining respective frame images of these plurality of videos, and outputs and displays the screen.

Another possible method is to sequentially generate, on a server, screens containing composite images which are each obtained by combining respective frame images of a plurality of videos and distribute the screens by streaming from the server to a client so that the client outputs and displays the screens.

Under the condition that the former method is employed, the client is required to perform processing of generating the screen by decoding each of the plurality of videos. However, if the client has low processing performance, there may occur a situation in which the client is incapable of the processing or takes a very long time to finish generating the screen. In addition, if the server and the client are connected by a network that is narrow in bandwidth, the former method may threaten to use up the bandwidth.

Under the condition that the latter method is employed, on the other hand, the server is required to execute processing of generating the screen. In the case of distributing a video from the server to many clients, for example, there may occur a situation in which the overall service level is lowered.

Another fact to consider is that, depending on the performance of the client and the performance of the server, letting the client display one or a few videos selected from a plurality of videos is sometimes better than letting the client display a plurality of videos at once.

Thus, with regard to the number of videos that constitute the basis of a screen presented to a user, using many videos is suitable in some cases and using a few videos is suitable in other cases.

The present invention has been made in view of the problems described above, and an object of the present invention is therefore to provide an image processing system, an image processing method, a video transmitting device, a video receiving device, an information storage medium, and a program with which flexibility is enhanced in setting the number of videos that are transmitted from a video transmitting device to a video receiving device to constitute the basis of a screen presented to a user.

In order to solve the problems described above, according to the present invention, there is provided a server, comprising: a plurality of a game emulator units; a composite image generating unit; and a video transmitting unit that transmits a video to a video receiving device; and a data receiving unit configured to receive from the video receiving device information indicating a first user's input, wherein the composite image generating unit generates a composite image of a plurality of game screens in different sizes generated by respective ones of the game emulator units, including a game emulator unit of the first user; the game emulating unit of the first user executes an update of game situation data in a manner that reflects the first user's input; and wherein the video transmitting unit transmits, to the video receiving device, the composite image.

According to the present invention, there is also provided an image processing method, comprising: an operating step of operating a plurality of a game emulator units at a server; a receiving step of receiving from a video receiving device information indicating a first user's input; a composite image generating step of generating, by a composite image generating unit of the server, a composite image of a plurality of game screens generated by respective ones of the game emulator units, including a game emulator unit of the first user; and executing an update of game situation data in a manner that reflects the first user's input; and a video transmitting step of transmitting, by a video transmitting unit of the server, the composite image to the video receiving device.

An embodiment of the present invention is hereinafter described in detail referring to the drawings.

<FIG> is a diagram illustrating an example of the overall configuration of a computer network according to the embodiment of the present invention. As illustrated in <FIG>, a cloud service <NUM> and clients <NUM> (<NUM>-<NUM> to <NUM>-n), which are all constructed based on computers, are connected to a computer network <NUM> such as the Internet. The cloud service <NUM> and the clients <NUM> can communicate to/from one another. In this embodiment, the cloud service <NUM> functions as a video transmitting device and the clients <NUM> each function as a video receiving device. The cloud service <NUM> and the clients <NUM> in this embodiment function as an image processing system on the whole.

Each of the clients <NUM> is a computer utilized by a user of the cloud service <NUM>, and is, for example, a personal computer, a game console, a television set, a portable game device, or a portable information terminal. The client <NUM> includes, for example, a control device such as a CPU, a memory device such as a memory element including a ROM or a RAM, or a hard disk drive, an output device such as a display or a speaker, an input device such as a game controller, a touch pad, a mouse, a keyboard, or a microphone, a communication device such as a network board, and an optical disc drive that reads data from an optical disc such as a digital versatile disc (DVD)-ROM or Blu-ray (trademark) disc (computer-readable information storage medium).

The clients <NUM> according to this embodiment each have a Web browser and a client program which are pre-installed therein, and execute these application programs.

The cloud service <NUM> is, for example, a distributed computing environment, and includes, among others, a plurality of Web application servers, a plurality of database servers, and a plurality of storage devices which are connected in a manner that allows communication to/from one another. The servers included in the cloud service <NUM> each include, for example, a control unit such as a CPU, a storage unit which is a ROM, a RAM, or other types of memory element, or a hard disk drive, and a communication unit which is a communication interface such as a network board. Those components are connected via a bus.

The clients <NUM> using the cloud service <NUM> can use various services provided by the cloud service <NUM> without being particularly conscious of the locations of servers, storage devices, and other resources in the cloud service <NUM>.

According to this embodiment, the clients <NUM> each access the cloud service <NUM> through the web browser, and input a user ID and a password. Then, under the condition that the client <NUM> accesses a predetermined URL, a screen corresponding to the predetermined URL is displayed on the display of the client <NUM>. After the entry of the user ID and the password, the cloud service <NUM> can identify the user ID of the user who utilizes the client <NUM> by, for example, referring to a cookie.

In this embodiment, under the condition that a user A, a user B, a user C, and a user D transmit, for example, a request to start executing a game in a mode allowing a plurality of users to play (multi-play) to the cloud service <NUM> from the respective clients <NUM>, the cloud service <NUM> starts executing the game. The user IDs of the user A, the user B, the user C, and the user D are hereinafter "<NUM>", "<NUM>", "<NUM>", and "<NUM>", respectively. For the duration of the execution of the game, the cloud service <NUM> transmits by streaming an encoded video that shows the specifics of this game play session to each of the clients <NUM>. In other words, the cloud service <NUM> sequentially transmits by streaming frame images <NUM> that show the specifics of this game play session to each of the clients <NUM>. Each client <NUM> that has received the video decodes the video, generates a game screen <NUM> in which the frame images <NUM> of the decoded video are arranged, and outputs the game screen <NUM> to the display in order to display the game screen <NUM> (see <FIG> and <FIG>). A video that includes a series of frame images <NUM> sequentially transmitted by streaming as described above to show the specifics of a game play session in progress is hereinafter referred to as play session video.

<FIG> illustrates an example of a game screen <NUM> that is displayed on the display of the client <NUM> of the user A. <FIG> illustrates an example of the game screen <NUM> that is displayed on the display of the client <NUM> of the user B. In the example of <FIG>, the frame image <NUM> of the play session video of the user B is placed in the upper left section, and the frame images <NUM> of play session videos of the user A, the user C, and the user D are placed in the upper right section, the lower left section, and the lower right section, respectively.

This embodiment allows each user to enjoy a game by pressing a button on the game controller or the like while viewing a play session video. In this embodiment, each user performs an operation of moving a player object or other operations in a game through key input such as pressing a button on the game controller.

As illustrated in <FIG> and <FIG>, there are cases in this embodiment where different game screens <NUM> are output to and displayed on the displays of the clients <NUM> of the respective users in multi-play in which the same game is played by a plurality of users. There are also cases in this embodiment where the number of play session videos transmitted from the cloud service <NUM> to the client <NUM> varies from one user to another in multi-play in which the same game is played by a plurality of users.

In this embodiment, under the condition that a user who is not playing the game (here, a user E, for example) transmits from his/her client <NUM> to the cloud service <NUM> a request to watch the game play session of the user A, the user B, the user C, and the user D, for example, the cloud service <NUM> distributes, for example, play session videos of the game played by the four users to the client <NUM> of the user E. The client <NUM> of the user E then outputs to the display the game screen <NUM> in which the frame images <NUM> of play session videos of the user A, the user B, the user C, and the user D are placed, for example, in the upper left section, the upper right section, the lower left section, and the lower right section, respectively, in order to display the game screen <NUM>. The user E can thus view play session videos that show the specifics of a game play session played by the four users in this embodiment.

<FIG> is a conceptual diagram illustrating an example of the relation between the cloud service <NUM> and the clients <NUM> according to this embodiment. The cloud service <NUM> according to this embodiment executes, for example, a management process <NUM>, a service providing process <NUM>, and emulators <NUM>. Image files of various game programs are stored in advance in storage devices and other memory devices that are included in the cloud service <NUM>. The game programs are each associated with a game ID which is the identifier of a game.

The service providing process <NUM> is, for example, a process that is generated by a server included in the cloud service <NUM> by activating a program that implements one of various services provided by the cloud service <NUM>, such as a shopping site and a social networking service (SNS).

The management process <NUM> is, for example, a process that is generated by a server included in the cloud service <NUM> by activating a management program which is installed in servers or storage devices included in the cloud service <NUM>. The management process <NUM> executes processing of managing the location of the emulator <NUM>, processing of activating the emulator <NUM>, processing of shutting down the emulator <NUM>, processing of connecting one of the clients <NUM> to the emulator <NUM> in response to a request from the client <NUM>, processing of disconnecting one of the clients <NUM> from the emulator <NUM> in response to a request from the client <NUM>, and the like.

The emulator <NUM> is, for example, a process that functions as a virtual machine for executing a game program stored in a server or a storage device that is included in the cloud service <NUM>. The emulator <NUM> is generated by the management process <NUM> by activating, in response to a request from one of the clients <NUM>, an emulator program installed in a server or a storage device that is included in the cloud service <NUM>. The management process <NUM> then reads image files of a game program specified by the client <NUM> in response to a request from the client <NUM>, and loads the image files onto the emulator <NUM>. This causes the game program to be executed on the emulator <NUM>. Alternatively, the emulator <NUM> may load a game program specified by the client <NUM> onto its own process.

In response to requests from the clients <NUM> or from the management process <NUM>, the emulator <NUM> outputs, for example, an emulated memory image or register (e.g., a program counter) that is managed by the emulator <NUM>, a log of input/output access made by a game program that runs on the emulator <NUM>, and a log of CPU commands or GPU commands executed on the emulator <NUM>.

This embodiment allows the exclusive use of one emulator <NUM> by one client <NUM> as illustrated in <FIG> (see a game program P1). This embodiment also allows a plurality of clients <NUM> to connect to one emulator <NUM> (see a game program P2). This embodiment also allows one emulator <NUM> to execute a plurality of game programs (see game programs P3 and P4).

In this embodiment, in the case where an emulator program is installed in each client <NUM> as illustrated in <FIG>, the management process <NUM> outputs an instruction to activate the emulator <NUM> or an instruction to shut down the emulator <NUM> to the client <NUM> in response to a request from the client <NUM>. The client <NUM> responds to the activation instruction received from the management process <NUM> by activating the emulator program that is installed in the client <NUM>. The client <NUM> responds to the shutdown instruction received from the management process <NUM> by shutting down the emulator <NUM> that is being run. This embodiment is also designed so that the emulator <NUM> running on the client <NUM> can execute a game program stored in the client <NUM> (see a game program P5). This embodiment is also designed so that another client <NUM> can be connected to the emulator <NUM> running on the client <NUM> (see a game program P6).

This embodiment is also designed so that one client <NUM> can be connected to a plurality of emulators <NUM> as illustrated in <FIG> (see game program P7 and P9).

<FIG> is a functional block diagram illustrating an example of functions that are implemented by the cloud service <NUM> and clients <NUM> according to this embodiment. <FIG> illustrates only functions that have particular relevance to the following description, and other functions than those of <FIG> are also implemented in the cloud service <NUM> and clients <NUM> according to this embodiment.

As illustrated in <FIG>, the cloud service <NUM> according to this embodiment includes, in terms of function, a data storage unit <NUM>, a data receiving unit <NUM>, a data transmitting unit <NUM>, a distribution management unit <NUM>, a game executing unit <NUM>, a composite image generating unit <NUM>, a game situation monitoring unit <NUM>, and a network situation monitoring unit <NUM>, for example. The data storage unit <NUM> is implemented mainly by, for example, a memory device such as a memory, a hard disk drive, or a storage device that is a component of a server included in the cloud service <NUM>. The data receiving unit <NUM> and the data transmitting unit <NUM> are implemented mainly by network boards or other communication interfaces included in the cloud service <NUM>. The game executing unit <NUM> corresponds to, for example, a function implemented by the emulator <NUM>. The rest of the components corresponds to, for example, functions implemented by the management process <NUM>.

As illustrated in <FIG>, the clients <NUM> according to this embodiment each include, in terms of function, a data receiving unit <NUM>, a data transmitting unit <NUM>, a distribution settings management unit <NUM>, a screen generating unit <NUM>, and a screen outputting unit <NUM>, for example. The data receiving unit <NUM> and the data transmitting unit <NUM> are implemented mainly by network boards or other communication interfaces included in the client <NUM>. The distribution settings management unit <NUM> and the screen generating unit <NUM> are implemented mainly by a CPU or other control devices included in the client <NUM>. The screen outputting unit <NUM> is implemented mainly by a display or other output devices included in the client <NUM>.

The cloud service <NUM> is built with a computer as a main component as described above, and the respective functional components of <FIG> that the cloud service <NUM> serves as are implemented by executing a program. The program is provided to the cloud service <NUM> via, for example, a computer-readable information storage medium such as a CD-ROM or a DVD-ROM, or a communication network such as the Internet. Further, the client <NUM> is also built with a computer as a main component as described above, and the respective functional components of <FIG> that the client <NUM> serves as are implemented by executing the above-mentioned client program. The program is provided to the client <NUM> via, for example, a computer-readable information storage medium such as a CD-ROM or a DVD-ROM, or a communication network such as the Internet.

The distribution management unit <NUM> in this embodiment performs, for example, the activation or shutdown of the emulator <NUM> in response to a request from one of the clients <NUM>. The distribution management unit <NUM> also executes processing of connecting one of the clients <NUM> to the emulator <NUM> in response to a request from the client <NUM>, and processing of disconnecting one of the clients <NUM> from the emulator <NUM> in response to a request from the client <NUM>. The distribution management unit <NUM> also loads image files of a game program onto the activated emulator <NUM>. The distribution management unit <NUM> further executes processing of managing the location of the emulator <NUM>.

In this embodiment, the data transmitting unit <NUM> of one client <NUM> transmits to the cloud service <NUM>, for example, a request to start executing an action game in a mode allowing a plurality of users to play (multi-play) that is associated with attribute data of that client <NUM>. The attribute data of each client <NUM> specifically indicates, for example, the name of the client <NUM>, the display size of the client <NUM>, the number of CPUs included in the client <NUM>, the type of a CPU included in the client <NUM>, the capacity of a hard disk drive included in the client <NUM>, the size of a memory included in the client <NUM>, the maximum communication speed of the client <NUM>, whether or not the client <NUM> includes a touch pad, and the type of the client <NUM> (which one of a personal computer, a game console, a television set, a portable game device, and a portable information terminal the client <NUM> is). In the example given here, the clients <NUM> of the user A, the user B, the user C, and the user D each transmit a request to start executing an action game in a multi-play mode that is associated with its own attribute data. These start requests are received by the data receiving unit <NUM> of the cloud service <NUM>.

The data storage unit <NUM> in this embodiment stores in advance attribute-settings association data <NUM>, an example of which is illustrated in <FIG>. The attribute-settings association data <NUM> associates attributes that are indicated by the attribute data described above with the screen configuration of the game screen <NUM> to be output to the client <NUM> in question, a method of distributing a video to the client <NUM>, and other settings. The attribute-settings association data <NUM> in the example of <FIG> associates a combination of display size data, which indicates the display size of the client <NUM>, and CPU count data, which indicates the number of CPUs included in the client <NUM>, with distribution target video data, which indicates at least one video to be distributed, composition necessity data, which indicates whether or not to combine the frame images <NUM> in the composite image generating unit <NUM>, and screen configuration data, which indicates the screen configuration of the game screen <NUM>.

The distribution target video data indicates, for example, whether to distribute a play session video of the user of the client <NUM> from which the game execution start request has been transmitted (hereinafter referred to as sender user) (this option is expressed as "sender user" in <FIG>), or play session videos of all players who play the game in a multi-play mode (this option is expressed as "all players" in <FIG>).

The term "screen configuration" in this embodiment refers to, for example, positions in the game screen <NUM> at which the frame images <NUM> of play session videos are placed (e.g., the position of each frame image <NUM> in the game screen <NUM> expressed in an X coordinate and a Y coordinate at which the upper left corner of the frame image <NUM> is placed), the size (e.g., the vertical and horizontal pixel counts of each frame image <NUM>), and the resolution. This screen configuration is indicated by screen configuration data in this embodiment.

In the attribute-settings association data <NUM> of this embodiment, a value "<NUM>-inch" of the display size data is associated with, for example, a value "type A" of the screen configuration data. A type-A screen configuration refers to, for example, a screen configuration in which the frame image <NUM> of the sender user's play session video is placed at the center as the one illustrated in <FIG>.

In the attribute-settings association data <NUM> of this embodiment, a combination of a value "<NUM>-inch" of the display size data and a value "<NUM>" of the CPU count data, and a combination of a value "<NUM>-inch" of the display size data and a value "<NUM>" of the CPU count data, are associated with, for example, a value "type B" of the screen configuration data. A type-B screen configuration refers to, for example, a screen configuration in which the frame image <NUM> of the sender user's play session video is placed in the upper left section, and the frame images <NUM> of other users' play session videos are placed at random in the upper right section, the lower right section, and the lower left section, as the one illustrated in <FIG>. The type-B screen configuration sets the frame image <NUM> of the sender user's play session video larger in size than the frame images <NUM> of the other users' play session videos. The screen configuration data may thus set different screen configurations to the frame image <NUM> of the sender user and to the frame images <NUM> of other users.

The distribution management unit <NUM> of the cloud service <NUM> generates distribution settings data <NUM> which indicates settings about video distribution and an example of which is illustrated in <FIG>. The distribution settings data <NUM> indicates, for example, the screen configuration of the game screen <NUM> at the client <NUM> in question, a play session video to be distributed, and a method of distributing a video such as whether or not to combine the frame images <NUM> of play session videos by the composite image generating unit <NUM> before transmitting to the client <NUM>.

The distribution settings data <NUM> in this embodiment includes, for example, the user ID of the user of the client <NUM> to which a video is distributed, a distributed video-associated user ID, which is the user ID of a user the specifics of whose play session are shown in the video to be distributed (one user ID or a plurality of user IDs may be set as the distributed video-associated user ID), composition method data, which indicates a method about combining the frame images <NUM> (in this embodiment, a value "distribute after composition" is set in the case of distributing the frame images <NUM> that are combined by the composite image generating unit <NUM>, and a value "distribute before composition" is set in the case of distributing a plurality of play session videos to the client <NUM> in question to generate the game screen <NUM> in the client <NUM> based on the plurality of frame images <NUM>), and screen configuration data, which indicates the screen configuration of the game screen <NUM>.

For example, the distribution management unit <NUM> in this embodiment identifies, for each client <NUM>, the distribution target video data, the composition necessity data, and the screen configuration data that are associated in the attribute-settings association data <NUM> with a combination of a display size and a CPU count that is included in the attribute data received from the client <NUM>. Based on these pieces of data, the distribution management unit <NUM> generates the distribution settings data <NUM> an example of which is illustrated in <FIG>. The distribution management unit <NUM> outputs the generated distribution settings data <NUM> to the data storage unit <NUM>.

This embodiment discusses an example in which the client <NUM> of the user A is a portable game terminal that is <NUM> inches in display size, the clients <NUM> of the user B, the user C, and the user D are game consoles that are <NUM> inches in display size, the CPU count is <NUM> in the clients <NUM> of the user A and the user B, and the CPU count is <NUM> in the clients <NUM> of the user C and the user D.

In this case, the distribution management unit <NUM> identifies "sender user" as the value of the distribution target video data, "unnecessary" as the value of the composition necessity data, and "type A" as the value of the screen configuration data, based on the fact that the attribute data received from the client <NUM> of the user A indicates <NUM> inches as the display size. Based on the identified values, the distribution management unit <NUM> generates the distribution settings data <NUM> that sets "<NUM>" as the value of the user ID, "<NUM>" as the value of the distributed video-associated user ID, and "type A" as the value of the screen configuration data.

The distribution management unit <NUM> identifies "all players" as the value of the distribution target video data, "necessary" as the value of the composition necessity data, and "type B" as the value of the screen configuration data, based on the fact that the attribute data received from the client <NUM> of the user B indicates <NUM> inches as the display size and <NUM> as the CPU count. Based on the identified values, the distribution management unit <NUM> generates the distribution settings data <NUM> that sets "<NUM>" as the value of the user ID, "<NUM>, <NUM>, <NUM>, <NUM>" as the value of the distributed video-associated user ID, "distribute after composition" as the value of the composition method data, and "type B" as the value of the screen configuration data.

The distribution management unit <NUM> identifies "all players" as the value of the distribution target video data, "unnecessary" as the value of the composition necessity data, and a value of the screen configuration data that corresponds to the game screen <NUM> of <FIG>, based on the fact that the attribute data received from the clients <NUM> of the user C and the user D indicates <NUM> inches as the display size and <NUM> as the CPU count. Based on the identified values, the distribution management unit <NUM> generates the distribution settings data <NUM> that sets "<NUM>" as the value of the user ID, "<NUM>, <NUM>, <NUM>, <NUM>" as the value of the distributed video-associated user ID, "distribute before composition" as the value of the composition method data, and "type B" as the value of the screen configuration data, and the distribution settings data <NUM> that sets "<NUM>" as the value of the user ID, "<NUM>, <NUM>, <NUM>, <NUM>" as the value of the distributed video-associated user ID, "distribute before composition" as the value of the composition method data, and "type B" as the value of the screen configuration data.

The data transmitting unit <NUM> of the cloud service <NUM> then transmits each piece of distribution settings data <NUM> to the client <NUM> that is identified by a user ID value contained in the piece of distribution settings data <NUM>. The data receiving unit <NUM> of each client <NUM> receives the distribution settings data <NUM>. The distribution settings management unit <NUM> of the client <NUM> outputs the received distribution settings data <NUM> to the memory device included in the client <NUM>.

The distribution management unit <NUM> of the cloud service <NUM> activates the emulators <NUM> and loads an image file of the action game onto the activated emulators <NUM>. The distribution management unit <NUM> connects the activated emulators <NUM> and the clients <NUM> of the user A, the user B, the user C, and the user D. The game executing unit <NUM> then starts executing the loaded action game. The cloud service <NUM> in this embodiment thus starts executing the action game in a multi-play mode in which the user A, the user B, the user C, and the user D participate as players.

In this embodiment, for a user whose composition method data has no value set in the distribution settings data <NUM>, or a user whose composition method data has a value "distribute before composition", the cloud service <NUM> executes three types of processing, game situation data updating processing, frame image generating processing, and frame image transmitting processing, in the order stated, for every given game-update-time (e.g., <NUM>/<NUM> seconds) since the start of the action game. The game situation data updating processing is executed by the game executing unit <NUM> to update game situation data which indicates the situation of a game. The frame image generating processing is executed by the game executing unit <NUM> to generate the frame images <NUM> that show the specifics of a game play session based on the updated game situation data. The frame image transmitting processing is executed by the data transmitting unit <NUM> to transmit the generated frame images <NUM> to the relevant client <NUM>. In short, the cloud service <NUM> repeatedly executes processing of sequentially executing the above-mentioned three types of processing at intervals of game-update-time.

In this embodiment, for a user whose composition method data has no value set in the distribution settings data <NUM>, or a user whose composition method data has a value "distribute after composition", the cloud service <NUM> executes four types of processing, game situation data updating processing, frame image generating processing, composite image generating processing, and frame image transmitting processing, in the order stated, for every given game-update-time (e.g., <NUM>/<NUM> seconds) since the start of the action game. The game situation data updating processing is executed by the game executing unit <NUM> to update game situation data which indicates the situation of a game. The frame image generating processing is executed by the game executing unit <NUM> to generate the frame images <NUM> that show the specifics of a game play session based on the updated game situation data. The composite image generating processing is executed by the composite image generating unit <NUM> to generate a composite image as the frame image <NUM> based on the frame images <NUM> generated by the frame image generating processing. The frame image transmitting processing is executed by the data transmitting unit <NUM> to transmit the frame image <NUM> generated by the composite image generating processing to the relevant client <NUM>. In short, the cloud service <NUM> repeatedly executes processing in which the four types of processing are executed sequentially at intervals of game-update-time. In short, the cloud service <NUM> repeatedly executes processing of sequentially executing the above-mentioned four types of processing at intervals of game-update-time.

In the frame image generating processing, the game executing unit <NUM> in this embodiment executes, for example, processing of generating frame images <NUM> based on game situation data that indicates the updated positions and directions of a group of the objects.

In an example of the composite image generating processing described above, the composite image generating unit <NUM> in this embodiment obtains the frame images <NUM> generated by the frame image generating processing, and generates a composite image based on the obtained frame images <NUM> so that the frame images <NUM> are placed in the same manner as a screen configuration that is indicated by the screen configuration data contained in the distribution settings data <NUM>. The frame image <NUM> to be transmitted to the client <NUM> of the user B in this embodiment is the frame image <NUM> generated by the composite image generating unit <NUM>. The composite image generating unit <NUM> generates, for example, a composite image in which the frame image <NUM> of the play session video of the user B is placed in the upper left section, and the frame images <NUM> of the play session videos of other users are placed in the upper right section, the lower left section, and the lower right section, as the frame image <NUM> to be transmitted to the client <NUM> of the user B.

In an example of the frame image transmitting processing, the data transmitting unit <NUM> in this embodiment transmits the generated frame image <NUM> to the relevant client <NUM> in association with a frame ID, which is an identifier assigned to each frame image <NUM> in the order of the time of generation, and the user ID of a user the specifics of whose play session are shown in the generated frame image <NUM>.

The data transmitting unit <NUM> of the cloud service <NUM> in this embodiment transmits the frame image <NUM> of a play session video in accordance with settings indicated by the distribution settings data <NUM>. To the client <NUM> of the user A, for instance, the data transmitting unit <NUM> of the cloud service <NUM> transmits the frame image <NUM> of a play session video that shows the specifics of a play session performed by the user A. To the client <NUM> of the user B, for instance, the data transmitting unit <NUM> of the cloud service <NUM> transmits the game screen <NUM> of <FIG> (the frame image <NUM> that corresponds to the game screen <NUM> of <FIG>). To the clients <NUM> of the user C and the user D, for instance, the data transmitting unit <NUM> of the cloud service <NUM> transmits the frame image <NUM> of a play session video that shows the specifics of a play session performed by the user A, the frame image <NUM> of a play session video that shows the specifics of a play session performed by the user B, the frame image <NUM> of a play session video that shows the specifics of a play session performed by the user C, and the frame image <NUM> of a play session video that shows the specifics of a play session performed by the user D. The data transmitting unit <NUM> of the cloud service <NUM> in this embodiment thus distributes a single play session video to each of the client <NUM> of the user A and the client <NUM> of the user B, and distributes four play session videos to each of the client <NUM> of the user C and the client <NUM> of the user D.

The data receiving unit <NUM> of each client <NUM> receives (obtains) the frame images <NUM> transmitted sequentially from the cloud service <NUM>.

The screen generating unit <NUM> of each client <NUM> generates the game screen <NUM> based on the distribution settings data <NUM> received from the cloud service <NUM> and the frame images <NUM> received from the cloud service <NUM>.

In this embodiment, the screen generating unit <NUM> of the client <NUM> of the user A, for example, generates the game screen <NUM> of <FIG>. The screen generating unit <NUM> of the client <NUM> of the user B generates the game screen <NUM> of <FIG> by placing at the center of the game screen <NUM> the frame image <NUM> corresponding to the game screen <NUM> of <FIG> which is transmitted from the cloud service <NUM>. In the clients <NUM> of the user C and the user D, the screen generating unit <NUM> arranges the respective frame images <NUM> in the game screen <NUM> based on the screen configuration data contained in the distribution settings data <NUM>, to thereby generate the game screen <NUM> similar to <FIG> which differs from <FIG> in terms of relation between the frame images <NUM> and sections where the frame images <NUM> are placed. In this embodiment, the screen generating unit <NUM> of the client <NUM> of the user C generates the game screen <NUM> that places the frame image <NUM> of a play session video of the user C in the upper left section, and the screen generating unit <NUM> of the client <NUM> of the user D generates the game screen <NUM> that places the frame image <NUM> of a play session video of the user D in the upper left section.

The screen output unit <NUM> of each client <NUM> outputs for display the game screen <NUM> generated by the screen generating unit <NUM> of its own client <NUM> to the display. Each client <NUM> in this embodiment generates, and outputs for display, the game screen <NUM> at a given frame rate. Each client <NUM> in this embodiment outputs for display the frame images <NUM> in the order of frame IDs associated with the frame images <NUM>. Play session videos are distributed by streaming from the cloud service <NUM> to the clients <NUM> in this manner.

As described above, in this embodiment, the number of play session videos distributed from the cloud service <NUM> to each of the client <NUM> of the user A and the client <NUM> of the user B is <NUM> and the number of play session videos distributed from the cloud service <NUM> to each of the client <NUM> of the user C and the client <NUM> of the user D is <NUM>.

Under the condition that the cloud service <NUM> is executing a game in which a plurality of users who respectively use a plurality of clients <NUM> participate in response to requests from the plurality of clients, this embodiment thus varies the screen configuration of the game screen <NUM> output for display by the clients <NUM> and the number of play session videos distributed to the clients <NUM>, depending on the display size and CPU count of the client <NUM> that the user uses. Whether the combining of the frame images <NUM> is executed by the cloud service <NUM> or by the client <NUM> also varies in this embodiment depending on the CPU count of the client <NUM> that the user uses, even in the case where the game screen <NUM> of the same screen configuration is output for display by each client <NUM>.

In the game situation updating processing, the game executing unit <NUM> in this embodiment executes processing of updating game situation data that indicates, for example, the positions and directions of player objects, opponent objects, and other character objects in a game. The game executing unit <NUM> in this embodiment also executes, in the game situation data updating processing, a collision detection processing between updated objects, game stage clear determining processing based on the result of the collision detection processing, miss determining processing, and the like.

In this embodiment, under the condition that one client <NUM> receives a key input made by the user such as the press of a button while the game is being executed, key information corresponding to the input (e.g., a signal indicating the type of the pressed button) is transmitted to the cloud service <NUM>. The game executing unit <NUM> of the cloud service <NUM> executes an update of game situation data in a manner that reflects the key information (e.g., to move a player object in a direction indicated by the pressed button).

The game situation monitoring unit <NUM> in this embodiment monitors, for example, the progress of game processing of a game that is being executed by the game executing unit <NUM>. The data storage unit <NUM> in this embodiment stores in advance the frame images <NUM> at points in time where given events occur in the game (to give a concrete example, the time when an encounter with a boss character occurs or the start point of a fight with a boss character). These frame images are hereinafter referred to as determination images. The game situation monitoring unit <NUM> in this embodiment monitors, for example, frame images generated by the game executing unit <NUM>. The game situation monitoring unit <NUM> uses a known image processing technology and, under the condition that it is confirmed that a monitored frame image matches one of the determination images stored in the data storage unit <NUM>, determines that an event such as an encounter with a boss character or the start of a fight with a boss character has occurred. The time when a given event occurs is thus detected in this embodiment.

Under the condition that an event is detected, the game situation monitoring unit <NUM> in this embodiment notifies the distribution management unit <NUM> of the fact. The distribution management unit <NUM> of the cloud service <NUM> updates the distribution settings data depending on the detected event.

Under the condition that the distribution management unit <NUM> in this embodiment receives from the game situation monitoring unit <NUM> a notification to the effect that an event has occurred in a game played by the user C, for example, the distribution management unit <NUM> changes the screen configuration data contained in the distribution settings data <NUM> that has "<NUM>", "<NUM>", and "<NUM>" as user ID values, so that the size of the frame image <NUM> of the user C is multiplied by a given number of times (e.g., <NUM> times).

The data transmitting unit <NUM> of the cloud service <NUM> transmits the changed distribution settings data <NUM> to the clients <NUM> of the user B, the user C, and the user D. In the client <NUM> of the user B, the client <NUM> of the user C, and the client <NUM> of the user D each, the distribution settings management unit <NUM> changes the distribution settings data <NUM> that has been stored in the memory device included in the client <NUM>. Under the condition that the distribution settings data <NUM> is changed in this embodiment, the cloud service <NUM> and the clients <NUM> execute processing based on the changed distribution settings data. For instance, the composite image generating unit <NUM> of the cloud service <NUM> and the screen generating unit <NUM> of each client <NUM> in this embodiment generate the game screen <NUM> based on the changed distribution settings data after the distribution settings data <NUM> is changed.

Under the condition that an event occurs in a game played by the user C in this embodiment, the area occupied by the frame image <NUM> of the play session video of the user C is thus expanded in each of the game screens <NUM> that are output to and displayed on the displays of the clients <NUM> of the user B, the user C, and the user D. This informs the users of the fact that an event in a game has occurred to the user C.

The network situation monitoring unit <NUM> in this embodiment monitors the communication situation (e.g., bandwidth) of the computer network <NUM> which connects the clients <NUM> and the cloud service <NUM>. Under the condition that the computer network <NUM> connecting the clients <NUM> and the cloud service <NUM> reaches a given upper limit bandwidth or higher, or reaches a given lower limit bandwidth or lower, the network situation monitoring unit <NUM> notifies the distribution management unit <NUM> of the fact. The distribution management unit <NUM> responds to this notification by, for example, changing the distribution settings data <NUM>.

Under the condition that the distribution management unit <NUM> in this embodiment is notified by the network situation monitoring unit <NUM> of the fact that the bandwidth has become lower than the given lower limit bandwidth in the computer network <NUM> connecting the cloud service <NUM> and the client <NUM> of the user C, for example, the distribution management unit <NUM> changes the value of the distributed video-associated user ID to "<NUM>", deletes the value of the composition method data, and changes the value of the screen configuration data to "type A" in the distribution settings data <NUM> that has "<NUM>" as the value of the user ID.

The data transmitting unit <NUM> of the cloud service <NUM> transmits the changed distribution settings data <NUM> to the client <NUM> of the user C. The client <NUM> of the user C changes the distribution settings data <NUM> that has been stored in the memory device included in the client <NUM>. In this manner, under the condition that the bandwidth becomes lower than the given lower limit bandwidth in the computer network <NUM> connecting the cloud service <NUM> and the client <NUM> of the user C in this embodiment, the number of the frame images <NUM> contained in the game screen <NUM> that is output for display on the display of the client <NUM> of the user C changes from four to one, and the number of play session videos transmitted from the cloud service <NUM> to the client <NUM> of the user C decreases from four to one as well.

It should be understood that, under the condition that the bandwidth becomes the given upper limit bandwidth or higher in the computer network <NUM> connecting the cloud service <NUM> and each client <NUM>, the cloud service <NUM> may increase the number of the frame images <NUM> contained in the game screen <NUM> that is output for display on the display of the client <NUM>, and may increase the number of play session videos transmitted from the cloud service <NUM> to the client <NUM> as well.

This embodiment is thus capable of changing the screen configurations of the game screens <NUM> output for display by the clients <NUM> and the numbers of play session videos transmitted from the cloud service <NUM> to the clients <NUM> under the condition that the situation changes with respect to the cloud service <NUM>, the clients <NUM>, and the computer network <NUM>, such as a change in the bandwidth of the computer network <NUM>. The cloud service <NUM> may also change the screen configurations of the game screens <NUM> output for display by the clients <NUM> and the numbers of play session videos transmitted from the cloud service <NUM> to the clients <NUM> in response to, for example, a change in load on the cloud service <NUM> or the clients <NUM>.

Under the condition that the distribution management unit <NUM> is notified by the network situation monitoring unit <NUM> of the fact that the bandwidth has become lower than the given lower limit bandwidth in the computer network <NUM> connecting the cloud service <NUM> and the client <NUM> of the user C, for example, the distribution management unit <NUM> may change the value of the composition method data to "distribute after composition" in the distribution settings data <NUM> that has "<NUM>" as the value of the user ID. After the distribution settings data <NUM> is changed, the cloud service <NUM> may transmit a composite image generated by the composite image generating unit <NUM> to the client <NUM> of the user C as the frame image <NUM>. This way, under the condition that the situation changes with respect to the cloud service <NUM>, the clients <NUM>, and the computer network <NUM>, such as a change in the bandwidth of the computer network <NUM>, which of the cloud service <NUM> and the client <NUM> is to execute the combining of the frame images <NUM> is switched without changing the screen configuration.

Under the condition that a user uses a touch pad or the like of the user's client <NUM> to execute an operation of changing the size of one frame image <NUM> or an operation of changing the display position of the frame image <NUM> in this embodiment, the data transmitting unit <NUM> of the client <NUM> transmits to the cloud service <NUM> a changing request to change the distribution settings data <NUM> that reflects the changing operation executed. The changing request is received by the data receiving unit <NUM> of the cloud service <NUM>. The distribution management unit <NUM> changes the distribution settings data <NUM> as requested by the received changing request. The data transmitting unit <NUM> of the cloud service <NUM> transmits the changed distribution settings data <NUM> to the client <NUM>. The distribution settings management unit <NUM> of the client <NUM> receives this distribution settings data <NUM> and changes distribution settings data that has been stored in the memory device. Under the condition that a user uses a mouse or the like to put (focus) a pointer on one frame image <NUM>, for example, the distribution management unit <NUM> of the cloud service <NUM> in this embodiment changes the distribution settings data <NUM> so that this frame image <NUM> is displayed large.

The present invention is not limited to the embodiment described above.

For example, under the condition that the user C performs a given operation with the use of a controller or the like while four users, the user A, the user B, the user C, and the user D, are playing a game in a multi-play mode as described above, the data transmitting unit <NUM> of the client <NUM> of the user C may transmit to the cloud service <NUM> a request to output for display the screen of the user C in a highlighted manner and the data receiving unit <NUM> of the cloud service <NUM> may receive this highlight output for display request. The distribution management unit <NUM> of the cloud service <NUM> may respond to the highlight output for display request by changing the distribution settings data <NUM> so that the frame image <NUM> of the play session video of the user C is displayed in a highlighted manner (e.g., displaying the frame image <NUM> of the user C enlarged, or putting a highlight image such as a closing line that makes the frame image <NUM> stand out around the frame image <NUM> of the user C). In this case, pieces of distribution settings data <NUM> that are stored in the memory devices included in the clients <NUM> of the user B, the user C, and the user D are also changed as described above. In the client <NUM> of the user B, the client <NUM> of the user C, and the client <NUM> of the user D each, the screen outputting unit <NUM> outputs for display the game screen <NUM> that displays the frame image <NUM> of the play session video of the user C in a highlighted manner to the display.

This way, when the user C wishes to draw the attention of other users to his/her play during a game played in a multi-play mode by four users, the user A, the user B, the user C, and the user D, for example, the user C can prompt other users to pay attention to his/her play.

The distribution target video data included in the attribute-settings association data <NUM> in association with given attributes may indicate, for example, the number of distribution target videos or an upper limit to the number of distribution target videos. To give a concrete example, "<NUM>" or "upper limit: <NUM>" may be set as the value of the distribution target video data. The screen configuration data included in the attribute-settings association data <NUM> may indicate the screen configuration of the game screen <NUM> in which as many frame images <NUM> as a number set in the distribution target video data are placed.

In the case where the attribute data received by the cloud service <NUM> indicates the given attributes described above, for example, the distribution management unit <NUM> of the cloud service <NUM> may then generate the distribution settings data <NUM> containing the distributed video-associated user ID that indicates as many user IDs as a distribution target video count (or upper limit count) indicated by the distribution target video data which are selected out of the users playing the game by following a given rule (e.g., the user IDs of users selected randomly from the sender user and other users), and the screen configuration data that indicates the screen configuration described above.

Under the condition that a game is played in a multi-play mode by four players, for example, the screen outputting unit <NUM> of each client <NUM> may output to the display the game screen <NUM> that contains the frame images <NUM> of play session videos of two users selected out of the four (i.e., that contains two frame images <NUM>).

The distribution management unit <NUM> of the cloud service <NUM> may generate the distribution settings data <NUM> that is suited to, for example, the load on servers that constitute the cloud service <NUM>, a predetermined network bandwidth, the current bandwidth of the computer network <NUM>, the delay time in video transmission, the jitter value, and the packet loss ratio in the computer network <NUM>.

In the case of video distribution from the cloud service <NUM> to a plurality of clients <NUM> (including the case where one video is distributed and the case where a plurality of videos are distributed), for example, the cloud service <NUM> may adjust the quality of streaming based on the load (CPU) on servers that constitute the cloud service <NUM>, the load on the bandwidth of the computer network <NUM>, and the priority level of each streaming distribution (e.g., the priority level based on whether or not a user who uses the client <NUM> is a member). For example, the cloud service <NUM> may lower the bit rate of streaming distribution or the image quality of a video for the client <NUM> of a user who is not a member.

The cloud service <NUM> may vary the bit rate of a video to be distributed or the encoding method depending on, for example, whether or not the client <NUM> in question includes a hardware decoder, whether or not a codec corresponding to the video is installed in the client <NUM>, video encoding load on the cloud service <NUM>, and other situations.

The distribution settings data <NUM> may include, for example, data other than the distributed video-associated user ID, the composition method data, and the screen configuration data, or may not include at least one of the distributed video-associated user ID, the composition method data, and the screen configuration data. The distribution settings data <NUM> may include, for example, a buffer size to be secured by the client <NUM> in question, an acceptable delay time, the frame rate, the bit rate, data indicating a distribution protocol (e.g., data indicating whether the distribution is by TCP or by UDP), and data indicating a communication path or the like. The data transmitting unit <NUM> of the cloud service <NUM> may distribute play session videos using a bit rate, a distribution protocol, a communication path, or the like that is indicated by the distribution settings data <NUM>. The client <NUM> may secure a buffer size indicated by the distribution settings data <NUM>. The screen generating unit <NUM> of the client <NUM> may generate a screen at a frame rate indicated by the distribution settings data <NUM>. The screen generating unit <NUM> of the client <NUM> may place the frame images <NUM> in the game screen <NUM> at a resolution indicated by the distribution settings data <NUM>.

The distribution management unit <NUM> may determine values that are indicated by the distribution settings data <NUM> based on factors other than the CPU count and the display size (e.g., the bandwidth of the computer network <NUM>, the load on the cloud service <NUM>, the name of the client <NUM> in question, the CPU type, the hard disk capacity, the memory size, the maximum communication speed, the presence or absence of a touch pad, and the type (which one of a personal computer, a game console, a television set, a portable game device, and a portable information terminal the client <NUM> is)).

The distribution management unit <NUM> may generate, for example, the distribution settings data <NUM> in which a resolution value suited to the size of the frame image <NUM> in question is set. In other words, there may be a correlation between the size of the frame image <NUM> and the resolution in the game screen <NUM>.

Each client <NUM> may output the game screen <NUM> to the display via a web browser, or may output the game screen <NUM> to the display as a screen generated by a client program.

The data storage unit <NUM> of the cloud service <NUM> may store data indicating a numerical expression for calculating a value that is indicated by the distribution settings data <NUM>. The distribution management unit <NUM> of the cloud service <NUM> may generate the distribution settings data <NUM> in which a set value calculated by the numerical expression is set.

The data transmitting unit <NUM> of the cloud service <NUM> may transmit, to each client <NUM>, as videos to be placed in one screen, videos in a number that is determined by, for example, a given rule different from the rules given above. The screen generating unit <NUM> of the client <NUM> may generate the game screen <NUM> that contains as many frame images <NUM> of videos as the determined number. The screen generating unit <NUM> of the client <NUM> may generate, for example, the game screen <NUM> in which a plurality of frame images <NUM> are placed at settings that follow a given rule different from the rules given above.

The embodiment described above may be applied to, for example, a case where a user watches how the same game is played in a multi-play mode by other users. In this case, the game screen <NUM> contains at least one frame image <NUM> of a play session video of another user. The processing described above may be applied to, for example, the case where a user is watching a racing game or the like played by a plurality of players. For instance, under the condition that an event such as a player setting a new lap time record, the race entering the final lap, or the finish line being crossed occurs, the distribution management unit <NUM> of the cloud service <NUM> may change the distribution settings data <NUM> so that the frame image <NUM> of a play session video of a user who has caused the event is enlarged.

The embodiment may also be applied to, for example, a case where a user watches a plurality of play session videos showing the specifics of play sessions of various games that are played by other users. In this case, the distribution management unit <NUM> of the cloud service <NUM> may generate the distribution settings data <NUM> based on the number of users currently viewing a play session video which is calculated from the number of clients <NUM> connected to the relevant emulator <NUM>, or based on the number of users who have viewed the play session videos which is calculated from the connection history of connection between the emulator <NUM> and the clients <NUM>. For example, the generated distribution settings data <NUM> may be set so that the frame image <NUM> of a play session video is larger in size under the condition that the number of users viewing the play session video is larger.

The embodiment may also be applied to, for example, a case where a user watches games being played by other users while enjoying playing a game himself/herself. In this case, the game screen <NUM> includes the frame image <NUM> of a play session video that shows the specifics of the user's own play and the frame images <NUM> of play session videos that show the specifics of other users' play. The distribution management unit <NUM> in this case may set the distribution settings data <NUM> so that the frame image <NUM> of the play session video that shows the specifics of the user's own play is larger in size than the frame images <NUM> of the play session videos that show the specifics of other users' play, or so that the frame image <NUM> of the play session video that shows the specifics of the user's own play is higher in image quality than the frame images <NUM> of the play session videos that show the specifics of other users' play. The distribution management unit <NUM> may also set the distribution settings data <NUM> so that the frame image <NUM> of a play session video of a game that is the same type or the same as the game that the user is playing is larger in size than the rest of the frame images <NUM>, or so that the frame image <NUM> of a play session video of a game that is the same type or the same as the game that the user is playing is higher in image quality than the rest of the frame images <NUM>. The screen generating unit <NUM> may thus generate the game screen <NUM> in which the frame images <NUM> are placed at settings that are determined by the relation between a game for which the specifics of a play session are shown in the frame image <NUM>, and a game that is being played by the user to whom the game screen <NUM> is presented. The screen generating unit <NUM> may also generate, for example, the game screen <NUM> in which the frame images <NUM> are placed so that the frame image <NUM> having a stronger relation between a game for which the specifics of a play session are shown in the frame image <NUM>, and a game that is being played by the user to whom the game screen <NUM> is presented is higher in image quality.

The embodiment described above may be applied not only to the distribution of a play session video but also to the case of distribution of a replay video, which is a video showing the specifics of a game play session that has already been executed. For instance, the data storage unit <NUM> of the cloud service <NUM> may store a replay video as a video showing the specifics of a game play session that has already been executed. The data transmitting unit <NUM> of the cloud service <NUM> may transmit the frame image <NUM> of the replay video to the clients <NUM>. In other words, the game screen <NUM> may include the frame image <NUM> of the replay video. The replay video may be a video encoded by Scalable Video Coding (SVC). The data transmitting unit <NUM> of the cloud service <NUM> may distribute the replay video to the clients <NUM> at an image quality and a resolution that conform to a distribution method indicated by the distribution settings data <NUM>.

The distribution management unit <NUM> may generate, for example, the distribution settings data <NUM> that indicates the priority levels and priority order of the frame images <NUM> contained in the game screen <NUM>. The screen generating unit <NUM> of each client <NUM> may arrange the frame images <NUM> in the game screen <NUM> based on the priority levels and the priority order that are indicated by the distribution settings data <NUM>. The distribution management unit <NUM> may generate, for example, the distribution settings data <NUM> for a user in which the priority level set to the frame image <NUM> of a play session video that shows the situation of a game played by the user is higher than the priority level of the frame image <NUM> of a play session video that shows the situation of a game watched by the user. Based on this distribution settings data <NUM>, the screen generating unit <NUM> of the client <NUM> may generate the game screen <NUM> so that in the game screen <NUM>, the frame image <NUM> of the play session video that shows the situation of the game played by the user is larger than the frame image <NUM> of the play session video that shows the situation of the game watched by the user.

The embodiment described above may be applied when, for example, a friend displaying screen <NUM>, which is a screen for displaying a list of users who are registered as users who have relevance to a user (hereinafter referred to as friends), is displayed as illustrated in <FIG>. The friend displaying screen <NUM> of <FIG> includes, for each friend who is executing a game in the cloud service <NUM>, an avatar image of the friend, the user ID of a user who is the friend, the game title of the game played by the user, the name of a game stage at which the user is playing, the total play time that the user has played the game, and the frame image <NUM> of the current play session video of the game played by the user. The friend displaying screen <NUM> can be scrolled. Under the condition that the friend displaying screen <NUM> is scrolled by a user, the data transmitting unit <NUM> of the client <NUM> of the user may transmit to the cloud service <NUM> a request to change the distribution settings data <NUM> so that only play session videos that are displayed in the friend displaying screen <NUM> are displayed by the client <NUM>. In response to this changing request, the cloud service <NUM> may change the distribution settings data <NUM> so that the frame images <NUM> to be displayed in the friend displaying screen <NUM> are transmitted to the client <NUM>. The system may also be designed so that, under the condition that a user clicks on the frame image <NUM> of a play session video at the client <NUM>, the user can view the play session video and participate in a game played in the play session video.

The cloud service <NUM> may transmit to the client <NUM>, for example, key information received from the client <NUM>, in association with data that indicates the time of the reception, instead of the frame image <NUM> of a play session video. The screen generating unit <NUM> of the client <NUM> may generate the game screen <NUM> based on the key information.

The data storage unit <NUM> may store in advance, for example, the specifics of an emulated memory image managed by the emulator <NUM> or a register value at the time when an event has occurred in a game, instead of the determination image. The game situation monitoring unit <NUM> may monitor the specifics of an emulated memory image managed by the emulator <NUM> or a register value to detect the occurrence of an event in a game based on the result of comparison between the result of the monitoring and the above-mentioned data stored in the data storage unit <NUM>.

To give yet another example, game situation data may be, for example, data that indicates a parameter or a status in a game. The functions implemented in the cloud service <NUM> may be implemented by, for example, a single server. The game program does not need to be executed on one of the emulators <NUM>. For instance, the game program may be executed by system software (an operating system or the like) of the cloud service <NUM> or the clients <NUM>. How the roles are divided between the cloud service <NUM> and the clients <NUM> is not limited to the example described above.

Claim 1:
A server (<NUM>), comprising:
a plurality of a game emulator units (<NUM>, <NUM>);
a composite image generating unit (<NUM>); and
a video transmitting unit (<NUM>) that transmits a video to a video receiving device (<NUM>); and
a data receiving unit (<NUM>) configured to receive from the video receiving device information indicating a first user's input,
wherein the composite image generating unit generates a composite image of a plurality of game screens in different sizes generated by respective ones of the game emulator units, including a game emulator unit of the first user;
the game emulating unit (<NUM>) of the first user executes an update of game situation data in a manner that reflects the first user's input; and
wherein the video transmitting unit transmits, to the video receiving device, the composite image.