A computer-readable tangible non-transitory storage medium storing a program causing a terminal to: display a first user interface object representing a gift of a first type on a display while video data related to a live-stream is reproduced; process data for realizing an effect for the gift represented by the first user interface object upon acceptance of selection of the first user interface object by a user of the terminal, the data being stored in a holding unit of the terminal; display a second user interface object representing a gift of a second type on the display while the video data is reproduced; and start download of data for realizing an effect for the gift represented by the second user interface object upon acceptance of selection of the second user interface object by the user.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims the benefit of priority from Japanese Patent Application Serial No. 2021-205415 (filed on Dec. 17, 2021), the contents of which are hereby incorporated by reference in its entirety.

TECHNICAL FIELD

The present disclosure relates to a terminal, a method, and a non-transitory computer-readable storage medium storing a program.

BACKGROUND

With the development of IT technology, the way information is exchanged has changed. In the Showa period (1926-1989), one-way information communication via newspapers and television was the main stream. In the Heisei period (1990-2019), with the widespread availability of cell phones and personal computers, and the significant improvement in Internet communication speed, instantaneous interactive communication services such as chat services emerged, and on-demand video distribution services also become popular as storage costs were reduced. And nowadays, with the sophistication of smartphones and further improvements in network speed as typified by 5G, services that enable real-time communication through video, especially live-stream services, are gaining recognition. The number of users of live-stream services is expanding, especially among young people, as such services allow people to share fun moments even when they are in the separate locations from each other.

Effects of gifts such as coins may help live-streamers engage more with viewers. If a wide variety of gift effects is offered and each gift can be elaborately designed, those gifts may make live-streams more exciting.

Japanese Patent Application Publication No. 2020-017870 describes a technique to make a live-stream more exciting by showing a Nagesen (tipping) effect with which a donation from a viewer to a live-streamer is shown on the live-stream screen.

Live-streaming requires immediate interactions between the distributor and the viewers. Therefore, usually all the data needed for gift effects is downloaded to a user’s terminal in advance so that they can be used immediately upon the user’s instruction.

However, the size of the data for realizing gift effects increases as the effects become more elaborate. Moreover, the more types of such gifts, the larger the data capacity required to store such gift data. Data of such a live-streaming application occupies a large portion of the storage capacity of the terminal in which the live-streaming application is installed, which may cause slow operation, limiting installation of other applications. This may lead to poor user experience.

SUMMARY

One object of the present disclosure is to provide a technique that enables the use of elaborate gifts in live-streaming while reducing the resulting increase in data volume.

One aspect of the disclosure relates to a non-transitory computer-readable storage medium storing a program. The program causes a terminal to: display a first user interface object representing a gift of a first type on a display while video data related to a live-stream is reproduced; process data for realizing an effect for the gift represented by the first user interface object upon acceptance of selection of the first user interface object by a user of the terminal, the data being stored in a holding unit of the terminal; display a second user interface object representing a gift of a second type on the display while the video data is reproduced; and start download of data for realizing an effect for the gift represented by the second user interface object upon acceptance of selection of the second user interface object by the user.

It should be noted that the components described throughout this disclosure may be interchanged or combined. The components, features, and expressions described above may be replaced by devices, methods, systems, computer programs, recording media containing computer programs, etc. Any such modifications are intended to be included within the spirit and scope of the present disclosure.

Advantageous Effects

According to the aspect of the present disclosure, it is possible to reduce the data volume increase in live-streaming while enabling the use of elaborate gifts.

DESCRIPTION OF EXAMPLE EMBODIMENTS

Like elements, components, processes, and signals throughout the Figures are labeled with same or similar designations and numbering, and the description for the like elements will not be hereunder repeated. For purposes of clarity and brevity, some of the components that are less related and thus not described are not shown in Figures.

Instead of downloading data that realize effects of all gifts (hereinafter referred to as “effect data”) to a user terminal, a live-streaming system related to an embodiment divides gifts into two types: a preloaded type gifts of which effect data is downloaded in advance, and a load-required type gifts of which effect data is downloaded when used. When a viewer wishes to use a gift that requires loading, the viewer taps the icon of the gift (hereinafter referred to as “gift icon”) to start downloading the effect data of the gift. Once the download is complete, the viewer can use the gift. Gift that are used infrequently may be configured as the load-required type gifts.

This can suppress the increase in the time required for preloading the gifts even if the size of each gift’s effect data increases, and can also suppress the increase in the total size of the preloaded effect data. In this way, the increase in communication performance and capacity required for user terminals can be suppressed while allowing richer animation and voice expression using larger effect data. In addition, since the preloaded type gifts do not require downloading when they are used, the immediacy of interaction between a distributor and viewers can be ensured for the preloaded type gifts.

As discussed above, the embodiment can reduce the download size of gifts by optimizing the gift download process, thereby achieving both a good user experience and a low load on the terminal.

Configuration of Live-Streaming System

FIG.1schematically illustrates a configuration of a live-streaming system 1 according one embodiment of the disclosure. The live-streaming system 1 provides an interactive live-stream service that allows a distributor LV (also referred to as a liver or streamer) and a viewer AU (also referred to as audience) (AU1, AU2...) to communicate in real time. As shown inFIG.1, the live-streaming system 1 includes a server10, a user terminal20on the distributor side, and user terminals30(30a,30b,) on the audience side. The distributor and viewers may be collectively referred to as users. The server10may be one or more information processing devices connected to a network NW. The user terminals20and30may be, for example, mobile terminal devices such as smartphones, tablets, laptop PCs, recorders, portable gaming devices, and wearable devices, or may be stationary devices such as desktop PCs. The server10, the user terminal20, and the user terminal30are interconnected so as to be able to communicate with each other over the various wired or wireless network NW.

The live-streaming system 1 involves the distributor LV, the viewers AU, and an administrator (not shown) who manages the server10. The distributor LV is a person who broadcasts contents in real time by recording the contents with his/her user terminal20and uploading them directly to the server 1. Examples of the contents may include the distributor’s own songs, talks, performances, fortune-telling, gameplays, and any other contents. The administrator provides a platform for live-streaming contents on the server10, and also mediates or manages real-time interactions between the distributor LV and the viewers AU. The viewer AU accesses the platform at his/her user terminal30to select and view a desired content. During live-streaming of the selected content, the viewer AU performs operations to comment and cheer via the user terminal30, the distributor LV who is delivering the content responds to such a comment and cheer, and such response is transmitted to the viewer AU via video and/or audio, thereby establishing an interactive communication.

The term “live-streaming” may mean a mode of data transmission that allows a content recorded at the user terminal20of the distributor LV to be played and viewed at the user terminals30of the viewers AU substantially in real time, or it may mean a live broadcast realized by such a mode of transmission. The live-streaming may be achieved using existing live streaming technologies such as HTTP Live Streaming, Common Media Application Format, Web Real-Time Communications, Real-Time Messaging Protocol and MPEG DASH. Live-streaming includes a transmission mode in which the viewers AU can view a content with a specified delay simultaneously with the recording of the content by the distributor LV As for the length of the delay, it may be acceptable for a delay even with which interaction between the distributor LV and the viewers AU can be established. Note that the live-streaming is distinguished from so-called on-demand type transmission, in which the entire recorded data of the content is once stored on the server and the data is then provided to a user at any subsequent time upon request from the user.

The term “video data” herein refers to data that includes image data (also referred to as moving image data) generated using an image capturing function of the user terminals20and30, and audio data generated using an audio input function of the user terminals20and30. Video data is reproduced in the user terminals20and30, so that the users can view contents.

FIG.2schematically illustrates an example of live-streaming implemented by the live-streaming system ofFIG.1. In the example inFIG.2, the distributor LV is live-streaming his/her talk. The user terminal20of the distributor LV generates video data by recording images and sounds of the distributor LV talking, and the generated data is transmitted to the server10(not shown inFIG.2) over the network NW. At the same time, the user terminal20displays a recorded video image VD of the distributor LV on the display of the user terminal20to allow the distributor LV to check the live stream currently performed.

The user terminals30a,30b, and30cof the viewers AU1, AU2, and AU3, respectively, who have requested the platform to view the live-stream of the distributor LV, receive video data related to the live-stream (may also be herein referred to as “live-streaming video data”) over the network NW and reproduce the received video data to display video images VD1, VD2, and VD3on the displays and output audio through the speakers. The video images VD1, VD2, and VD3displayed at the user terminals30a,30b, and30c, respectively, are substantially the same as the video image VD captured by the user terminal20of the distributor LV, and the audio outputted at the user terminals30a,30b, and30cis substantially the same as the audio recorded by the user terminal20of the distributor LV

Recording of the images and sounds at the user terminal20of the distributor LV and reproduction of the video data at the user terminals30a,30b,30cof the viewers AU1, AU2, AU3are performed substantially simultaneously. Once the viewer AU1types a comment about the talk of the distributor LV on the user terminal30a, the server10displays the comment on the user terminal20of the distributor LV in real time and also displays the comment on the user terminals30a,30b, and30cof the viewers AU1, AU2, and AU3, respectively. When the distributor LV read the comment and develops his/her talk to cover and respond the comment, the video and sound of the talk are displayed on the user terminals30a,30b,30cof the viewers AU1, AU2, AU3respectively. This interactive action is recognized as establishment of a conversation between the distributor LV and the viewer AU1. In this way, the live-streaming system 1 realizes the live-streaming that enables the interactive communication, not one-way communication.

FIG.3is a block diagram showing functions and configuration of the user terminal20ofFIG.1. The user terminal30has the same functions and configuration as the user terminal20. Each block inFIG.3and the subsequent block diagrams may be realized by elements such as a computer CPU or a mechanical device in terms of hardware, and can be realized by a computer program or the like in terms of software. Functional blocks realized by cooperative operation between these elements. Therefore, it is understood by those skilled in the art that these functional blocks can be realized in various forms by combining hardware and software.

The distributor LV and the viewers AU download and install a live-streaming application program (hereinafter referred to as a live-streaming application) according to the embodiment to the user terminals20and30from a download site over the network NW. Alternatively, the live-streaming application may be pre-installed on the user terminals20and30. When the live-streaming application is executed on the user terminals20and30, the user terminals20and30communicate with the server10over the network NW to implement various functions. Hereinafter, the functions implemented by the user terminals20and30(processors such as CPUs) in which the live-streaming application is run will be described as functions of the user terminals20and30. These functions are realized in practice by the live-streaming application on the user terminals20and30. In any other embodiments, these functions may be realized by a computer program that is written in a programming language such as HTML (HyperText Markup Language), transmitted from the server10to web browsers of the user terminals20and30over the network NW, and executed by the web browsers.

User terminal20has a delivery unit100that generates video data recording the user’s image and sound and provides it to server10, a viewing unit200that acquires and plays the video data from server10, a terminal-side gift holding unit250that holds information on gifts at user terminal20, and a download queue holding unit252that holds a queue for downloading gifts. The user activates the distribution unit100when the user performs live-streaming, and activates the viewing unit200when the user views a video. The user terminal in which the distribution unit100is activated is the distributor’s terminal, i.e., the user terminal that generates the video data, and the user terminal in which the viewing unit200is activated is the viewer’s terminal, i.e., the user terminal is which the video data is reproduced and played.

The distribution unit100includes an image capturing control unit102, an audio control unit104, a video transmission unit106, and a distributor-side UI control unit108. The image capturing control unit102is connected to a camera (not shown inFIG.3) and controls image capturing performed by the camera. The image capturing control unit102obtains image data from the camera. The audio control unit104is connected to a microphone (not shown inFIG.3) and controls audio input from the microphone. The audio control unit104obtains audio data through the microphone. The video transmission unit106transmits video data including the image data obtained by the image capturing control unit102and the audio data obtained by the audio control unit104to the server10over the network NW. The video data is transmitted by the video transmission unit106in real time. That is, the generation of the video data by the image capturing control unit102and the audio control unit104, and the transmission of the generated video data by the video transmission unit106are performed substantially at the same time.

The distributor-side UI control unit108controls an UI for the distributor. The distributor-side UI control unit108is connected to a display (not shown inFIG.3), and displays a video on the display by reproducing the video data that is to be transmitted by the video transmission unit106. The distributor-side UI control unit108displays an operation object or an instruction-accepting object on the display, and accepts inputs from the distributor who taps on the object.

The viewing unit200includes a viewer-side UI control unit202, a superimposed information generation unit204, an input information transmission-reception unit206, a gift determination unit208, and a queue control unit210. The viewer-side UI control unit202controls the UI for the viewers. The viewing-side UI control unit202is connected to a display and speaker (not shown inFIG.3), and reproduces the received video data to display video images on the display and output audio through the speaker. The state where the image is outputted to the display and the audio is outputted from the speaker can be referred to as “the video data is played”. The viewer-side UI control unit202is also connected to input means (not shown inFIG.3) such as touch panels, keyboards, and displays, and obtains user input via these input means. The superimposed information generation unit204superimposes a predetermined frame image on an image generated from the video data from the server10. The frame image includes various user interface objects (hereinafter simply referred to as “objects”) for accepting inputs from the user, comments entered by the viewers, and information obtained from the server10.

The viewer-side UI control unit202displays gift icons in conjunction with the superimposed information generation unit204during reproduction of the live-stream video data. The gift icons include a gift icon representing the preloaded type gift and a gift icon representing the load-required type gift.

The gift information transmission-reception unit206transmits and receives information related to gifts to and from the server10. The gift determination unit208determines the type of a gift. The queue control unit210controls the download queue holding unit252.

FIG.4is a data structure diagram showing an example of a terminal-side gift holding unit250ofFIG.3. The terminal-side gift holding unit250holds information on gifts that the user terminal20has downloaded. A gift is electronic data with the following characteristics:It can be purchased in exchange for the points (later described in detail), or can be given for free.It can be given by a viewer to a distributor. Giving a gift to a distributor is referred to as using the gift or throwing the gift.Some gifts may be purchased and used at the same time, and some gifts may be purchased or given and then used at any time later by the purchaser or recipient viewer.When a viewer gives a gift to a distributor, the distributor is awarded the amount of points corresponding to the gift and an effect associated with the gift is exerted. For example, an effect corresponding to the gift will appear on the live-streaming screen.

The terminal-side gift holding unit250stores a gift ID for identifying a gift, data of an icon that is an object representing the gift (hereinafter referred to as icon data), data for realizing an effect corresponding to the gift (hereinafter referred to as “effect data”), and last modified date, which is the date when the gift was last used, in association with each other. A viewer is able to present a desired gift to a distributor by paying the price or value equivalent to the gift while viewing the live-stream. The payment of the equivalent value may be made by an appropriate electronic payment means. For example, the payment may be made by the viewer paying points corresponding to the equivalent value to the administrator. Alternatively, bank transfers or credit card payments may be used.

The effect is a visual or auditory or tactile effect (e.g., vibration) or a combination thereof that characterizes a gift. Examples of the visual effect include animation, images, and flashing/blinking. Examples of the auditory effect include sound effects and voice. The effect data is data for realizing such an effect on the user terminal20, and the user terminal20realizes such an effect by processing the effect data. Since the technique for realizing the effect data itself is known, it will not be hereunder described in detail.

The gift IDs and icon data of all gifts are stored in advance in the terminal-side gift holding unit250. The gift information transmission-reception unit206downloads the effect data of gifts from the server10and stores it in the terminal-side gift holding unit250. There are some gifts for which the gift information transmission-reception unit206has not downloaded their effect data from the server10.

FIG.5is a data structure diagram showing an example of the download queue holding unit252ofFIG.3. The download queue holding unit252holds queues for gift downloads. The download queue holding unit252holds the download order and the gift ID in association with each other. The gift information transmission-reception unit206refers to the download queue holding unit252, sequentially requests the effect data of the gifts from the server10according to the order of the queue, and downloads the effect data from the server10.

FIG.6is a block diagram showing functions and configuration of the server ofFIG.1. The server10has a distribution information providing unit302, a relay unit304, a gift information providing unit306, a gift processing unit308, a gift download list310, a stream DB312, a user DB314, and a gift DB316.

FIG.7is a data structure diagram showing an example of the gift download list310ofFIG.6. The gift download list310is a list that specifies gifts for which the user terminals20and30must download the effect data by default. The effect data of the gifts with the gifts ID that are included in the gift download list310is automatically downloaded to the user terminals20and30when the live-streaming application is opened on the user terminals20and30. Specifically, the effect data is downloaded without the user’s instruction and stored in the terminal-side gift holding unit250. The gift download list310has an item of region, and gifts to be downloaded are listed for each region to which the user terminal belongs.

The gift download list310stores the gift IDs of the gifts to be downloaded in association with the regions. The gift download list310is configured to be updatable. The administrator of the live-streaming system is able to change the contents of the gift download list310via the server10. The administrator may register gifts (event-related gifts, etc.) that are expected to be used frequently in the gift download list310in advance. In addition, the administrator may remove infrequently used gifts from the gift download list310.

FIG.8is a data structure diagram of an example of the stream DB312ofFIG.6. The stream DB312holds information regarding a live-stream currently taking place. The stream DB312stores a stream ID for identifying a live- stream on a live distribution platform provided by the live-streaming system 1, a distributor ID for identifying the distributor who provides the live-stream, and a viewer ID for identifying a viewer of the live-stream, in association with each other.

FIG.9is a data structure diagram showing an example of the user DB314ofFIG.6. The user DB314holds information regarding users. The user DB314stores a user ID for identifying a user, points that the user has, and the level of the user, in association with each other. The point is the electronic value circulated within the live-streaming platform. When a distributor receives a gift from a viewer during a live-stream, the distributor’s points increase by the value of the gift. The points are used, for example, to determine the amount of reward or money the distributor receives from the administrator of the live-streaming platform. The level is an indicator of the amount of user activity on the live-streaming platform. The level of the user is raised by giving gifts as a viewer, performing live-streams as a distributor, participating in events, and the like. The server10calculates the level of the user from the history of the user’s activity.

FIG.10is a data structure diagram showing an example of the gift DB316ofFIG.6. The gift DB316stores the icon data and the effect data of all gifts. The gift DB316stores a gift ID for identifying a gift, the amount of points that are granted to a distributor when the gift is given to the distributor, the icon data for the gift, and the effect data for the gift, in association with each other.

Referring again toFIG.6, Upon reception of a notification from the user terminal20on the distributor side to start a live-stream over the network NW, the distribution information providing unit302registers a stream ID for identifying this live-stream and the distributor ID of the distributor who performs the live-stream in the stream DB312. When the distribution information providing unit302receives a request to provide information about live-streams from the viewing unit200of the user terminal30on the viewer side over the network NW, the distribution information providing unit302retrieves currently available live-streams from the stream DB312and makes a list of the available live-streams. The distribution information providing unit302transmits the list to the requesting user terminal30over the network NW. The viewer-side UI control unit202of the requesting user terminal30generates a live-stream selection screen based on the received list and displays it on the display of the user terminal30.

Once the viewer-side UI control unit202of the user terminal30receives the viewer’s selection result of the live-stream on the live-stream selection screen, the input information transmission unit206generates a distribution request including the stream ID of the selected live-stream, and transmits the request to the server10over the network NW. The distribution information providing unit302starts providing, to the requesting user terminal30, the live-stream specified by the stream ID included in the received distribution request. The distribution information providing unit302updates the stream DB312to include the viewer ID of the viewer of the requesting user terminal30into the viewer IDs of the stream ID.

The relay unit304relays the transmission of the video data from the distributor-side user terminal20to the viewer-side user terminal30in the live-streaming started by the distribution information providing unit302. The relay unit304receives from the viewer-side UI control unit202a signal that represents user input by the viewer during reproduction of the video data on the viewer-side user terminal30. The signal representing user input includes a gift DL request signal for requesting download of the effect data of a gift and a gift usage signal representing the use of a gift. The gift DL request signal includes the gift ID of the gift to be downloaded. The gift usage signal includes the viewer ID of a viewer, the distributor ID of a distributor to whom a gift is given (the distributor ID of the distributor performing the live-stream that is being viewed by the viewer who gave the item), and the gift ID of the gift.

The gift information providing unit306provides information on gifts to the user terminals20and30. The gift information providing unit306transmits, to the user terminal, the gift download list for the terminal in response to a request from the user terminals. The gift information providing unit306obtains, from the gift DB316, the effect data corresponding to the gift ID included in the gift DL request signal that has been received by the relay unit304. The gift information providing unit306transmits the obtained effect data to the requesting user terminal as a response to the gift DL request signal.

The gift processing unit308updates the user DB314so as to increase the points of the distributor depending on the points of the gift identified by the gift ID included in the gift usage signal. Specifically, the gift processing unit308refers to the gift DB316to specify the points to be granted for the gift ID included in the received gift usage signal. The gift processing unit308then updates the user DB314to add the determined points to the points of the distributor ID included in the gift usage signal.

The operation of the live-streaming system 1 with the above configuration will be now described.FIG.11is a flowchart showing steps of an application activation process on the user terminals20and30. Once a user taps the icon of the live-streaming application, the live-streaming application is opened on the user terminal (S502). The user terminal determines whether a lightweight mode is turned ON (S504). The lightweight mode is a mode configuring all the gifts for which effect data is not stored in the terminal-side gift holding unit250as the load-required type gifts at the time of start of the application. The user terminal holds the ON/OFF state of the lightweight mode by a mode holding unit (not shown). The user terminal switches ON/OFF of the lightweight mode according to an instruction from the user. When the lightweight mode is ON (YES in S504), the user terminal ends the application startup process without updating the download queue.

When the lightweight mode is OFF (NO in S504), the gift information transmission-reception unit206requests the gift download list from the server10(S506). The gift information transmission-reception unit206generates a list request signal including the user ID of the user of the user terminal and the region to which the user terminal belongs, and transmits the list request signal to the server10over the network NW The gift information providing unit306determines whether the attribute of the user with the user ID that is included in the received list request signal satisfies a predetermined criterion. When the criterion is met, the gift information providing unit306generates a gift download list for the terminal such that the list includes all gift IDs registered in the gift DB316. Specifically, the gift information providing unit306refers to the user DB314and specifies the level for the user ID included in the received list request signal. When the specified level exceeds a threshold value, the gift information providing unit306generates the gift download list for the terminal such that the list includes all gift IDs registered in the gift DB316. When the criterion is not met, the gift information providing unit306refers to the gift download list310and generates a gift download list for the terminal such that the list includes the gift IDs for the region included in the received list request signal. The gift information providing unit306transmits the generated gift download list for the terminal to the user terminal over the network NW. The gift information transmission-reception unit206of the user terminal20receives the gift download list for the terminal (S508).

The gift determination unit208deletes expired effect data from the terminal-side gift holding unit250based on the received gift download list for the terminal and the last modified date of each gift (S509). The gift determination unit208refers to the terminal-side gift holding unit250, and identifies a gift ID with which the difference between the last modified date and the current date exceeds a threshold value (for example, 14 days). The gift determination unit208determines whether the specified gift ID is included in the received gift download list for the terminal. When the specified gift ID is not included in the gift download list for the terminal, the gift determination unit208deletes the effect data for the gift ID from the terminal-side gift holding unit250. The gift determination unit208does not delete the specified gift ID when the specified gift ID is included in the gift download list for the terminal. It can be said that the gift determination unit208deletes the effect data of the gifts that satisfy the predetermined deletion criterion and are not included in the gift download list for the terminal. In this way, it is possible to suppress an increase in the size of the terminal-side gift holding unit250while ensuring the immediacy of the interaction by maintaining the effect data of the preloaded type gifts.

The gift determination unit208specify a gift(s) whose effect data should be downloaded based on the received terminal-side gift download list and the contents of the terminal-side gift holding unit250(S510). For each gift ID included in the terminal-side gift download list, when the effect data for the gift ID is not held in the terminal-side gift holding unit250, the gift-determining unit208specifies the gift ID as the gift ID of the gift to be downloaded. The gift determination unit208registers the specified gift IDs in the download queue holding unit252in a predetermined or any order (S512). The gift information transmission-reception unit206starts downloading such effect data in the order that the effect data is registered in the download queue holding unit252(S514).

In the above flow, the gift information transmission-reception unit206starts downloading the effect data for the preloaded type gifts, that is, the effect data for the gifts included in the gift download list for the terminal, before icons of the gifts are specified without an instruction from the user. The gift determination unit208identifies the preloaded type gifts by referring to the gift download list for the terminal.

When the user’s attributes meet the predetermined criterion in step S506, the gift download list for the terminal includes the gift IDs of all the gifts, so that the gift information transmission-reception unit206starts downloading the effect data for the gifts regardless of the gift type without any instruction from the user.

In the above flow, since the gift IDs of the load-required type gifts are not included in the gift download list for the terminal, the download of effect data for a load-required type gift is not started before this gift is specified by the user. However, this does not apply when the gift is used by another user as described later.

FIG.12is a flowchart showing steps of a download process performed on the user terminals20and30. The gift information transmission-reception unit206of the user terminal starts downloading the effect data of the gift specified by the gift ID at the top of the queue in the download queue holding unit252(S520). The gift information transmission-reception unit206generates a gift DL request signal that includes the highest gift ID among the gift IDs held in the download queue holding unit252, that is, the first gift ID in the queue, and transmits the gift DL request signal to the server10over the network NW.

The queue control unit210updates the download queue holding unit252by deleting the gift ID at the top of the queue in the download queue holding unit252and raising the order of the remaining gift IDs by one (S522). For example, in the download queue holding unit252, the second gift ID is raised to the first or top of the queue, and the third gift ID is raised to the second in the queue.

The gift information transmission-reception unit206determines whether the download is completed (S524). When completed (YES in S524), the gift information transmission- reception unit206registers the downloaded effect data in the terminal-side gift holding unit250, and the process returns to step S520. When not completed (NO in S524), the queue control unit210determines whether a queue change event has occurred (S526). There may be two following causes to change the queue.

(Cause 1) The user has indicated his/her intention to use a load-required type gift or a preloaded type gift that has not been downloaded yet. Specifically, an icon of such a gift was tapped by the user.

(Cause 2) Another viewer of another user terminal to which the same live-stream video data is streamed has used a load-required type gift or a preloaded type gift that has not been downloaded yet.

When there is no such a cause occurred to change the queue (NO in S526), the process returns to step S524. When such a queue change cause occurs (YES in S526), the queue control unit210updates the download queue holding unit252depending on the queue change cause (S528). The process then returns to step S524.

Step S528will be described in detail. Cause 1 will be described later with reference toFIG.13. Cause 2 will be described with reference to the example ofFIG.2. The distributor LV distributes the live-stream, and the viewer AU1and the viewer AU2are viewing the same live-stream. Here, when the viewer AU2uses a gift, a gift usage signal that includes the gift ID of the gift is generated by the user terminal30band transmitted to the server10over the network NW At the same time, the user terminal30bprocesses effect data for the gift to superimpose and display the effect on the video image VD2. By referring to the stream DB312, the server10identifies another viewer(s) who is (are) viewing the same live-stream as the live-stream that the viewer AU2is viewing. The server10generates a gift notification signal that includes the gift ID included in the received gift usage signal, and transmits the gift notification signal to the user terminal30aof the specified another viewer (here, the viewer AU1) over the network NW. The user terminal30adetermines whether the effect data for the gift ID included in the received gift notification signal is held in the terminal-side gift holding unit250. When the effect data is held, the user terminal30aprocesses the effect data to superimpose and display the same effect as the effect of the user terminal30bon the video image VD1. Whereas when the effect data is not held, the user terminal30bdetermines that Cause 2 has occurred.

The queue control unit210controls the download queue holding unit252such that the following priority relationship is established for the gift downloads.

First Priority: Download effect data for a gift related to Cause 1.

Second Priority: Download effect data for a gift related to Cause 2.

Third Priority: Download effect data for a gift registered in the download queue holding unit252when the live-streaming application is started.

When Cause 1 occurs, the queue control unit210moves down the order of all the gift IDs held in the download queue holding unit252by one, and inserts the gift ID of the gift related to Cause 1 in the first place or top of the queue. That is, the queue control unit210wedges the gift ID of the gift related to Cause 1 to the top of the download queue. For example, in the download queue holding unit252, the first gift ID is descended to the second in the queue, the second gift ID is descended to the third in the queue, and the gift ID of the gift related to Cause 1 is registered first in the queue. As a result, download of the effect data of the gift related to the Cause 1 will be started following the download currently in progress.

When Cause 2 occurs, the queue control unit210moves down the order of all the gift IDs held in the download queue holding unit252by one except for the gift ID of the gift related to Cause 1. In addition, the gift ID of the gift related to Cause 2 is inserted into the entry directly under the gift ID of the gift related to Cause 1. That is, the queue control unit210interrupts the download queue by inserting the gift ID of the gift related to Cause 2 between the gift ID of the gift related to Cause 1 and the other gift IDs in the download queue. For example, in the download queue holding unit252, the gift ID related to Cause 1 whose order was the first remains in the first place in the queue, the gift ID whose order was the second is moved to the third in the queue, and the gift related to Cause 2 is registered in the second place in the queue.

FIG.13is a flowchart showing steps of a gift usage process on the user terminals20and30. In the example, it is assumed that the viewer accesses the live-stream platform from the user terminal30, selects a desired live-stream on the live-stream selection screen, and starts viewing the selected live-stream. While the viewer is watching the live-stream, the video data is continuously transmitted from the distributor’s user terminal20to the viewer’s user terminal30via (the relay unit304of) the server10.

During the reproduction of the video data, the viewer performs user input for requesting display of an item via input means of the user terminal30. Upon reception of this user input, the viewing-side UI control unit202of the user terminal30refers to the terminal-side gift holding unit250and obtains the download status of each gift (S280). The download status includes “download completed” when the effect data is held in the terminal-side gift holding unit250, “not downloaded” when the effect data is not held in the terminal-side gift holding unit250, and “now downloading” when the download of the effect data is currently in progress.

The viewer-side UI control unit202and the superimposed information generation unit204display each gift icon in a manner corresponding to the download status obtained in step S280(S282). The superimposed information generation unit204generates a gift icon for each gift by obtaining icon data from the terminal-side gift holding unit250and processing the data. The superimposed information generation unit204imparts, to the generated gift icon, a visual effect according to the download status obtained in step S280. For example, when the download status is “now downloading”, the superimposed information generation unit204associates an indicator indicating the download status with the corresponding gift icon. For example, the superimposed information generation unit204adds a progress bar indicating the progress of download to the gift icon. When the download status is “not downloaded”, the superimposed information generation unit204determines whether the gift is in the download queue by referring to the download queue holding unit252. When the gift is in the download queue, the superimposed information generation unit204gives, to the corresponding gift icon, a visual effect (shading, etc.) indicating that the gift is waiting for download. When the gift is not in the download queue, the superimposed information generation unit204put a mark to the corresponding gift icon to indicate that download is necessary. When the download status is “download completed”, the superimposed information generation unit204does not give a visual effect to the corresponding gift icon. The superimposition information generation unit204superimposes the processed or unprocessed gift icon thus generated on the image of the video data obtained from the server10. The viewer-side UI control unit202displays the image on which the gift icon is superimposed on the display.

The viewing side UI control unit202waits until it detects tapping on the gift icon shown on the display in step S282(S284). If the download status is changed during waiting (for example, “now downloading” turns to “download completed”, or the queued “not downloaded” turns to “now downloading” and then turns to “download completed”), the visual effect of the gift icon also changes as the download status changes. When tapping on the gift icon is detected (YES in S284), the gift determination unit208determines the download status of the gift corresponding to the gift icon specified through the tapping, “not downloaded”, “download completed”, or “now downloading” (S286).

When the download status is “not downloaded”, the queue control unit210inserts the gift ID of the gift specified by tapping the corresponding icon into the top of the queue in the download queue holding unit252(S288). This corresponds to the case where the above Cause 1 occurs. Thereafter the process returns to step S280. As a result, download of the effect data of the gift specified by tapping is started with the highest priority.

When the download status is “now downloading”, the viewer UI control unit202displays, on the display, a pop-up that includes text indicating that the effect data of the gift specified by tapping is being downloaded. The text may be, for example, “Now loading. When loading is complete, tap the icon to present the gift.” The process then returns to step S280.

When the download status is “DL completed”, the gift information transmission-reception unit206generates a gift usage signal that includes the gift ID of the gift specified by tapping, and transmits the gift usage signal to the server10over the network NW (S292). The superimposed information generation unit204and the viewer-side UI control unit202outputs the gift effect by reading the gift effect data specified by tapping from the terminal-side gift holding unit250and processing it (S294). The gift determination unit208accesses the terminal-side gift holding unit250and updates the last modified date of the gift specified by tapping to the current date (S296). The process then returns to step S280.

FIG.14is a representative screen image of the live-stream selection screen602displayed on the display of the viewer user terminal30. The live-stream selection screen602includes thumbnails604indicating live-streams in the list of currently available live streams. The viewer-side UI control unit202generates the live-stream selection screen602based on the list of live-streams obtained from the server10and shows the screen on the display.

FIG.15is a representative screen image of a live-streaming room screen610shown on the display of the viewer user terminal30. Once the viewer taps a thumbnail on a live-stream selection screen602ofFIG.14, the live-streaming room screen610ofFIG.15is shown on the display. The live-streaming room screen610includes a distributor image612obtained by reproducing the video data, and gift icons614,616, and618.

The gift icon614is a gift icon for a gift whose download status is “not downloaded” and which is not registered in the download queue holding unit252. The gift icon614has a mark620indicating that download is required. The gift represented by this gift icon614is a load-required type gift.

The gift icon616is a gift icon for a gift whose download status is “not downloaded” and which is registered in the download queue holding unit252. The gift icon616is shaded to indicate that it is waiting for download. The gift represented by this gift icon616is basically a preloaded type gift that has not yet been downloaded.

The gift icon618is a gift icon for a gift whose download status is “download completed”. The gift represented by this gift icon618is: (1) a preloaded type gift for which effect data has been downloaded; and (2) a load-required type gift for which effect data has been downloaded as the user uses the gift or the gift is specified by other user. Since the effect data of the gift (2) has already been downloaded, it is no longer the load-required type gift. However, when the corresponding effect data has expired and the gift is deleted from the terminal-side gift holding unit250, the gift again becomes the load-required type gift.

Once the viewer taps the gift icon618on the live-streaming room screen610ofFIG.15, the user terminal30accepts the designation of the special gift icon618by the viewer. The user terminal30realizes the effect for the gift icon618by reading effect data for the gift represented by the designated gift icon618from the terminal-side gift holding unit250and processing the effect data.

FIG.16is a representative screen image of a live-streaming room screen622shown on the display of the viewer user terminal30. Once the viewer taps the gift icon614on the live-streaming room screen610ofFIG.15, the user terminal30accepts the selection of the gift icon614by the viewer. The user terminal30then starts downloading the effect data for the gift represented by the specified gift icon614(hereinafter referred to as a user-designated gift). In the live-streaming room screen622ofFIG.16, the gift icon614ofFIG.15is changed to a gift icon 626 that includes a progress bar624indicating the progress of the download instead of the mark620.

FIG.17is a representative screen image of a live-streaming room screen628shown on the display of the viewer user terminal30. Since the time has passed from the state ofFIG.15and the download of the effect data of the user-specified gift has been completed, the gift icon630of the user-specified gift on the live-streaming room screen628ofFIG.16is shown as the gift icon whose download status is “download completed”. Download of the effect data for the gift represented by the gift icon616on the live-streaming room screen610ofFIG.15is now in progress, the gift icon632of the gift includes the progress bar.

FIG.18is a representative screen image of a live-streaming room634shown on the display of the viewer user terminal30. Once the viewer taps the gift icon630on the live-streaming room screen628ofFIG.17, the user terminal30displays the live-streaming room screen634that includes a distributor image636on which the effect of the user-specified gift is superimposed on the display.

FIG.19is a representative screen image of a live-streaming room screen638shown on the display of the viewer user terminal30. The lightweight mode is ON on the live-streaming distribution screen638. Since there are no preloaded type gifts in the lightweight mode, gift icons640,642, and644displayed on the live-streaming room screen638all have the download status of “not downloaded” and the corresponding gifts are not registered in the download queue holding unit252. The gift icons each have a mark indicating that download is required. The gifts represented by these gift icons are load-required type gifts.

In the above embodiment, an example of the holding unit includes a hard disk or semiconductor memory. It is understood by those skilled in the art that each element or component can be realized by a CPU not shown, a module of an installed application program, a module of a system program, or a semiconductor memory that temporarily stores the contents of data read from the hard disk, and the like.

The live-streaming system 1 in the embodiment does not download the effect data for all the gifts in advance but download the effect data for the specified preloaded-type gifts in advance. Download of effect data for the remaining load-required type gifts is triggered by the user’s specification of the load-required type gift(s). In this way, it is possible to suppress an increase in the amount of communication caused by downloading and an increase in the capacity required for installing the live-streaming application while realizing a richer effect by using a large effect data.

Further, in the live-streaming system1 in the embodiment, when a viewer taps the preloaded type gift, its effect is immediately exerted on the screen of the distributor, the screen of the viewer, and the screens of another viewers. Therefore, the immediacy of the interaction between the distributor and the viewers can be maintained.

In one example, the total amount of effect data is several tens of gigabytes (GB). By applying the technical idea of the present embodiment, the amount of effect data held by the user terminal can be reduced to several gigabytes to several hundred megabytes (MB). Further, it takes several tens of milliseconds to several seconds to download a piece of typical effect data in a typical communication environment, but the adverse effect on the user experience due to such a delay is limited. Therefore, in many cases, the advantages of providing the load-required type outweigh the disadvantages.

Further, in the live-streams system 1 in the embodiment, the gifts that have been downloaded and the gifts that have not been downloaded yet are displayed in different modes. Therefore, the user can tell at a glance the download status of the gifts, which improves user convenience. In particular, as a characteristic of live-streaming, there are often situations where a user wishes to present a gift immediately on the spot, and in such a situation, the user can easily select a downloaded gift while avoiding the gifts that have not been downloaded yet. This improves user experience.

Further, in the live-streaming system 1 in the embodiment, download of the effect data for the gift specified by the user has priority over the automatic download of the preloaded type gifts. In this way, it is possible to reduce the time that the user has to wait for using a gift, and thus it is possible to increase the satisfaction of the user.

Further, in the live-streaming system 1 in the embodiment, download of the effect data for the gift used by other user has a higher priory than the automatic download of the preloaded type gifts. In this way, the delay in the effect output can be reduced, so that it is possible to reduce the unpleasant feeling caused by, for example, gap of timing between when the distributor says thank you and when the effect is exerted.

Further, in the live-streaming system 1 in the embodiment, all gifts are preloaded for users whose attributes satisfy a predetermined criterion. This allows, for example, to maximize the user experience of users who have been using the live distribution platform for a long time and / or who have a high total billing amount by preloading all the gifts. Alternatively, since the user terminals of so-called heavy users are likely to have high functionality and large capacity, the user experience can be maximized by preloading all gifts to such users.

Hardware Configuration Example

Referring toFIG.20, the hardware configuration of the information processing device will be now described.FIG.20is a block diagram showing an example of a hardware configuration of the information processing device according to the embodiment. The illustrated information processing device900may, for example, realize the server10and the user terminals20and30in the embodiment.

The information processing device900includes a CPU901, ROM (Read Only Memory)903, and RAM (Random Access Memory)905. The information processing device900may also include a host bus907, a bridge909, an external bus911, an interface913, an input device915, an output device917, a storage device919, a drive921, a connection port925, and a communication device929. In addition, the information processing device900includes an image capturing device such as a camera (not shown). In addition to or instead of the CPU901, the information processing device900may also include a DSP (Digital Signal Processor) or ASIC (Application Specific Integrated Circuit).

The CPU901functions as an arithmetic processing device and a control device, and controls all or some of the operations in the information processing device900according to various programs stored in the ROM903, the RAM905, the storage device919, or the removable recording medium923. For example, the CPU901controls the overall operation of each functional unit included in the server10and the user terminals20and30in the embodiment. The ROM903stores programs, calculation parameters, and the like used by the CPU901. The RAM905serves as a primary storage that stores a program used in the execution of the CPU901, parameters that appropriately change in the execution, and the like. The CPU901, ROM903, and RAM905are interconnected to each other by a host bus907which may be an internal bus such as a CPU bus. Further, the host bus907is connected to an external bus911such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge909.

The input device915may be a user-operated device such as a mouse, keyboard, touch panel, buttons, switches and levers, or a device that converts a physical quantity into an electric signal such as a sound sensor typified by a microphone, an acceleration sensor, a tilt sensor, an infrared sensor, a depth sensor, a temperature sensor, a humidity sensor, and the like. The input device915may be, for example, a remote control device utilizing infrared rays or other radio waves, or an external connection device927such as a mobile phone compatible with the operation of the information processing device900. The input device915includes an input control circuit that generates an input signal based on the information inputted by the user or the detected physical quantity and outputs the input signal to the CPU901. By operating the input device915, the user inputs various data and instructs operations to the information processing device900.

The output device917is a device capable of visually or audibly informing the user of the obtained information. The output device917may be, for example, a display such as an LCD, PDP, or OELD, etc., a sound output device such as a speaker and headphones, and a printer. The output device917outputs the results of processing by the information processing unit900as text, video such as images, or sound such as audio.

The storage device919is a device for storing data configured as an example of a storage unit of the information processing equipment900. The storage device919is, for example, a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, or an optical magnetic storage device. This storage device919stores programs executed by the CPU901, various data, and various data obtained from external sources.

The drive921is a reader/writer for a removable recording medium923such as a magnetic disk, an optical disk, a photomagnetic disk, or a semiconductor memory, and is built in or externally attached to the information processing device900. The drive921reads information recorded in the mounted removable recording medium923and outputs it to the RAM905. Further, the drive921writes record in the attached removable recording medium923.

The connection port925is a port for directly connecting a device to the information processing device900. The connection port925may be, for example, a USB (Universal Serial Bus) port, an IEEE1394 port, an SCSI (Small Computer System Interface) port, or the like. Further, the connection port925may be an RS-232C port, an optical audio terminal, an HDMI (registered trademark) (High-Definition Multimedia Interface) port, or the like. By connecting the external connection device927to the connection port925, various data can be exchanged between the information processing device900and the external connection device927.

The communication device929is, for example, a communication interface formed of a communication device for connecting to the network NW. The communication device929may be, for example, a communication card for a wired or wireless LAN (Local Area Network), Bluetooth (trademark), or WUSB (Wireless USB). Further, the communication device929may be a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), a modem for various communications, or the like. The communication device929transmits and receives signals and the like over the Internet or to and from other communication devices using a predetermined protocol such as TCP / IP. The communication network NW connected to the communication device929is a network connected by wire or wirelessly, and is, for example, the Internet, home LAN, infrared communication, radio wave communication, satellite communication, or the like. The communication device929realizes a function as a communication unit.

The image capturing device (not shown) is an imaging element such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor), and a device that captures an image of the real space using various elements such as lenses for controlling image formation of a subject on the imaging element to generate the captured image. The image capturing device may capture a still image or may capture a moving image.

The configuration and operation of the live-streaming system 1 in the embodiment have been described. This embodiment is a merely example, and it is understood by those skilled in the art that various modifications are possible for each component and a combination of each process, and that such modifications are also within the scope of the present disclosure.

The technical idea according to the embodiment may be applied to live commerce or virtual live-streaming using an avatar that moves in synchronization with the movement of the distributor instead of the image of the distributor.

In the embodiment, when using the load-required type gift, first tap the gift icon of the gift to start downloading the effect data. After the download is completed, tap the gift icon again to use the gift. However, the embodiment is not limited to this. For example, when the user terminal detects tapping on a load-required type gift icon, it starts downloading the effect data for the gift represented by the gift icon. Upon completion of the download, processing for use of the gift may be automatically performed without further instruction from the user. In this case, the number of taps for using the gift is reduced, which reduces the user’s effort.

In the embodiment, the case where the download of the effect data of the load-required type gift is started when tapping on the gift icon of the load-required type gift is detected has been described. However the embodiment is not limited to this. For example, once the user terminal receives a request from the user to display the gift icon of a load-required type gift via the input means, the user terminal may start downloading the effect data for the gift.

In the embodiment, when the user has changed the region to which the user terminal belongs, the user terminal may delete the effect data for the gift(s) that is not included in the gift download list of the new region from the terminal-side gift holding unit250.

In the embodiment, the case where the user terminal compares the gift download list with the contents of the terminal-side gift holding unit250has been described. However the embodiment is not limited to this. For example, when the live-streaming application is started, the user terminal refers to the terminal-side gift holding unit250to generate a list of gift IDs of gifts for which effect data exists and sends the list to the server10. The server10compares the received list with the gift download list to specify the effect data to be downloaded to the user terminal. The server10transmits the gift ID(s) of the specified effect data to the user terminal. The user terminal registers the received gift ID (s) in the download queue holding unit252and starts downloading.

In the embodiment, when Cause 1 or Cause 2 occurs, the download of the effect data for the gift related to the cause is started after the ongoing download is completed. However the invention is not limited to this. For example, when Cause 1 or Cause 2 occurs, the user terminal may suspend or cancel the currently ongoing download and start downloading the effect data for the gift related to the cause.

The procedures described herein, particularly those described with a flow diagram, a flowchart, are susceptible of omission of part of the steps constituting the procedure, adding steps not explicitly included in the steps constituting the procedure, and/or reordering the steps. The procedure subjected to such omission, addition, or reordering is also included in the scope of the present invention unless diverged from the purport of the present invention.

At least some of the functions realized by the server10may be realized by a device(s) other than the server10, for example, the user terminals20and30. At least some of the functions realized by the user terminals20and30may be realized by a device(s) other than the user terminals20and30, for example, the server10. For example, the superimposition of a predetermined frame image on an image of the video data performed by the user terminal where the video data is reproduced may be performed by the server10or may be performed by the user terminal where the video data is generated.