Patent ID: 12231716

DESCRIPTION OF EMBODIMENTS

Embodiments of the present disclosure are described below in detail with reference to the drawings. Note that, in the drawings, the same reference signs are assigned to the same or equivalent constituent elements. In addition, although, in the following description, a case where a distribution server (server device) identifies a comment (comment to which a streamer reacted) is described, the present disclosure can likewise be applied to even a case where a streamer terminal identifies a comment, as described later. That is, the embodiments to be described below are for explaining the present disclosure, not for limiting the scope of the present disclosure. Accordingly, it is possible for a person skilled in the art to employ embodiments in which part or all of the elements of the embodiments are replaced by equivalents thereof, which are also included in the scope of the present disclosure.

Embodiment 1

FIG.1is a schematic diagram illustrating an example of an overall configuration of a distribution system1according to Embodiment 1 of the present disclosure. In the distribution system1, a distribution server100that distributes a live video, a streamer terminal200that is used by a streamer, and a viewer terminal300that is used by a viewer are connected to one another in a communicable manner via the Internet900, as an example. Note that it is assumed that a large number of streamer terminals200and a large number of viewer terminals300exist according to the number of streamers and the number of viewers who uses the system, respectively.

The distribution server100is, for example, a server device (computer used as a server or the like) and distributes a live video provided by a streamer to a viewer terminal300. The distribution server100also sends a comment posted during distribution of a live video to a streamer terminal200and a viewer terminal300.

Each streamer terminal200is a terminal, such as a personal computer (PC), a smartphone, and a tablet, and is used by a streamer who performs live distribution. For example, the streamer captures a live video for introducing a product. By the streamer terminal200providing the distribution server100with the live video, the live video is distributed from the distribution server100to a viewer terminal300. Note that the streamer is able to, during distribution of a live video, browse comments posted by viewers using viewer terminals300and, reacting to a predetermined comment, refer to the comment in the live video.

Each viewer terminal300is a terminal, such as a smartphone, a tablet, and a PC, and is used by a viewer who views a distributed live video. For example, the viewer starts a video viewing app (or a browser app) in the viewer terminal300and, after logging in to the distribution server100and the like, views a live video distributed from the distribution server100. In addition, the viewer is able to post a comment addressed to the streamer of a live video that the viewer is viewing. Specifically, on a screen of the video viewing app or the like, a video area for displaying a live video and a comment area for displaying a comment are formed, as described later. Note that the comment area is configured such that latest comments can be displayed in a predetermined number. In other words, while the number of comments does not reach the predetermined number, every time a comment is posted, the comment is successively added to the comment area. After the number of comments has reached the predetermined number, when a comment is posted, an older comment is removed from the comment area in a last-in-first-out manner and the new comment is added to the comment area.

(Outline Configuration of Information Processing Device400)

A typical information processing device400by which the distribution server100, each of the streamer terminals200, and each of the viewer terminals300according to Embodiment 1 are achieved is described.

The information processing device400includes, as illustrated inFIG.2, a central processing unit (CPU)401, a read only memory (ROM)402, a random access memory (RAM)403, a network interface card (NIC)404, an image processor405, a sound processor406, an auxiliary storage407, an interface408, an image capturing unit409, a voice unit410, an operation unit411, and a display unit412.

The CPU401controls operation of the entire information processing device400, and is connected to the respective constituent elements and exchanges control signals and data with the constituent elements.

In the ROM402, an initial program loader (IPL), which is executed immediately after power-on, is recorded, and, by the IPL being executed, programs stored in the auxiliary storage407are read into the RAM403and execution of the programs by the CPU401is started.

The RAM403is a memory device for temporarily recording data and programs, and, in the RAM403, programs and data read from the auxiliary storage407and, in addition thereto, data required for communication and the like are stored.

The NIC404is a device for connecting the information processing device400to a computer communication network, such as the Internet, and includes a circuit conforming to the 10BASE-T/100BASE-T standard that is used when a local area network (LAN) is constructed, an analog modem for establishing a connection to the Internet using a telephone line, an integrated services digital network (ISDN) modem, an asymmetric digital subscriber line (ADSL) modem, a cable modem for establishing a connection to the Internet using a cable television line, or the like.

The image processor405processes image data read from the auxiliary storage407or the like, using an image calculation processor (not illustrated) that the CPU401or the image processor405includes and subsequently records the processed image data in a frame memory (not illustrated) that the image processor405includes. Image data recorded in the frame memory are converted to a video signal at a predetermined synchronization timing, and the video signal is output to the display unit412via the interface408and the like. In other words, under the control of the CPU401, the image processor405generates an image required in advancing processing performed by the information processing device400and causes the display unit412to display the image.

The sound processor406converts music data and voice data read from the auxiliary storage407or the like to a sound signal and outputs the sound signal to the outside via the interface408and the like. Note that, when the information processing device400includes a speaker thereinside, the sound processor406outputs the converted sound signal to the speaker. In other words, under the control of the CPU401, the sound processor406generates a music sound and a voice required to be generated in advancing processing performed by the information processing device400and causes the music sound and the like to be output from an internal or external speaker.

The auxiliary storage407is a hard disk, a solid state drive (SSD), or the like and stores various types of programs and a various types of data required for operation control of the entire information processing device400. For example, in the auxiliary storage407, programs for achieving the distribution server100and the like according to the embodiments are stored. Under the control of the CPU401, the auxiliary storage407appropriately reads stored programs and data and causes the RAM403or the like to temporarily store the read programs and data.

The interface408conforms to a standard, such as HDMI (registered trademark), universal serial bus (USB), and inter-integrated circuit (I2C), and, to the interface408, the image capturing unit409, the voice unit410, the operation unit411, and the display unit412are connected. Note that the interface408may send and receive necessary information to and from a connected external device in addition to the units.

The image capturing unit409includes, for example, a single focus lens and an imaging element (a complementary metal oxide semiconductor (CMOS) or a charge coupled device (CCD)) of a predetermined number of pixels and captures a still image or a video.

The voice unit410is, for example, a microphone or the like and accepts as input a voice (speech) of a worker or the like who uses the information processing device400.

The operation unit411accepts operation input from a worker or the like who uses the information processing device400.

The display unit412draws an image corresponding to image data output by the image processor405and presents the image to a worker or the like who uses the information processing device400.

In addition, the information processing device400may be configured to include, in place of the auxiliary storage407, a drive unit of a digital versatile disc (DVD)-ROM or the like. In this case, the information processing device400reads programs and data from a DVD-ROM or the like mounted on the drive unit and operates in the same manner as described above.

Functions and the like of the distribution server100, each of the streamer terminals200, and each of the viewer terminals300, which are achieved by the above-described information processing device400, are described below with reference toFIGS.3to8. When the information processing device400is powered on, programs that cause the information processing device400to function as the distribution server100, the streamer terminal200, or the viewer terminal300according to Embodiment 1 are executed, and the distribution server100, the streamer terminal200, or the viewer terminal300according to Embodiment 1 is achieved.

(Functional Configuration of Distribution Server100)

FIG.3is a block diagram illustrating an example of a functional configuration of the distribution server100according to Embodiment 1. As illustrated, the distribution server100includes a sender110, a receiver120, a storage130, and a controller140.

The sender110sends various types of information addressed to a viewer terminal300and the like via the Internet900. For example, the sender110distributes a live video provided by a streamer terminal200to a viewer terminal300. The sender110also sends a comment posted from a viewer terminal300to a streamer terminal200and a viewer terminal300. Further, the sender110sends command information to change a display form of a comment to a viewer terminal300and the like. Note that the afore-described NIC404can function as the sender110as described above.

The receiver120receives various types of information sent from a streamer terminal200and a viewer terminal300via the Internet900. For example, the receiver120receives a live video provided by a streamer terminal200. The receiver120also receives a comment posted from a viewer terminal300. Note that the afore-described NIC404can function as the receiver120as described above.

The storage130stores various types of information required for processing in the distribution server100. For example, the storage130stores comment management information131.

The comment management information131is information to manage a comment posted by a viewer during distribution of a live video and includes, as an example, information as illustrated inFIG.4. Specifically, in the comment management information131, pieces of information, such as a comment ID131a, a reception time131b, a comment131c, a display flag131d, and a reaction flag131e, are included. Note that the display flag131dis a flag indicating whether or not the comment is currently displayed in a comment area for displaying a predetermined number of comments in a viewer terminal300. Specifically, when the display flag131dis “1”, the display flag131dindicates that the comment is a comment to be displayed, and, in contrast, when the display flag131dis “0”, the display flag131dindicates that the comment is not a comment to be displayed. In addition, the reaction flag131eis a flag indicating whether or not the streamer reacted to the comment. Specifically, when the reaction flag131eis “1”, the reaction flag131eindicates that the streamer reacted to the comment, and, in contrast, when the reaction flag131eis “0”, the reaction flag131eindicates that the streamer has not reacted (unreacted) to the comment.

The storage130stores, in addition to the comment management information131, various types of information required for processing performed by the controller140. For example, the storage130stores an ID, a password, and the like of a viewer who uses a viewer terminal300. The storage130also stores, as information relating to a live video, a video management ID, a video title, a video summary, a distribution start time, and the like. Note that the afore-described auxiliary storage407or the like can function as the storage130as described above.

Returning toFIG.3, the controller140controls the entire distribution server100. The controller140includes, for example, a distribution controller141, a comment manager142, a speech recognizer143, an identifier144, and a display controller145.

The distribution controller141distributes a live video received from a streamer terminal200to a viewer terminal300. That is, when the receiver120receives a live video provided by a streamer terminal200, the distribution controller141controls the sender110to distribute the received live video to a viewer terminal300.

The comment manager142manages a comment posted by a viewer who is viewing a live video. That is, when the receiver120receives a comment posted from a viewer terminal300, the comment manager142controls the sender110to send the received comment to a streamer terminal200and a viewer terminal300. In addition, the comment manager142updates the comment management information131, illustrated in the above-describedFIG.4. Specifically, the comment manager142adds a newly posted comment to the comment management information131(sets “1” and “0” in the display flag131dand the reaction flag131eof the comment, respectively) and also changes the display flag131dof, among comments the display flags131dof which are “1”, a comment the reception time131bof which is the oldest to “0”. In addition, when the identifier144, to be described later, identifies a comment, the comment manager142changes the reaction flag131eof the comment to “1”.

The speech recognizer143subjects speech spoken by a streamer in a live video to speech recognition in real time and converts the recognized speech to a speech text. For example, every time the streamer speaks, the speech recognizer143segments voice data into respective phoneme units, identifies each phoneme through comparison with feature distributions of sample phonemes, subjects the identified phonemes to pattern matching with words, and the like, and subsequently finally generates a speech text representing speech that the streamer spoke.

Every time a voice of the streamer is converted to a speech text by the speech recognizer143, the identifier144, by comparing the converted speech text with each comment that is currently displayed in the comment area of a viewer terminal300, identifies a comment to which the streamer reacted (a comment to which the streamer referred in the live video). More specifically, the identifier144first subjects a speech text (natural sentence) to morphological analysis and extracts a keyword (for example, a noun or a noun phrase). Next, the identifier144, referring to the comment management information131illustrated inFIG.4, subjects each comment the display flag131dof which is “1” to morphological analysis and extracts keywords of the respective comments. The identifier144compares the keyword extracted from the speech text (the keyword included in the speech text) with the keywords respectively extracted from the respective comments (the keywords respectively included in the respective comments) and identifies a comment including a keyword coinciding with the keyword in the speech text as a comment to which the streamer reacted.

More specifically, a case where the streamer speaks “Driving time of the battery was roughly four hours.” is described. Note that it is assumed that, at a point of time of this speech, comments (comments131c) the display flags131dof which are “1” in the comment management information131are “Your explanation is excellent.”, “I see.”, “I prefer the previous type.”, “I've been waiting for that function.”, and “How long is the driving time?”. In this case, the identifier144extracts keywords “battery”, “driving time”, and “four hours” from the speech text converted by the speech recognizer143, and extracts keywords “explanation is excellent”, “-(no keyword)”, “previous type”, “function”, and “driving time” from the respective comments. The identifier144compares each keyword extracted from the speech text with the respective keywords extracted from the respective comments. Since, in this case, the keyword “driving time” in the speech text and the keyword “driving time” in one of the comments coincide with each other, the identifier144identifies the comment “How long is the driving time?”, which includes the keyword “driving time”, as a comment that the streamer reacted.

Note that the identifier144may text-search the respective comments the display flags131dof which are “1” with each keyword extracted from the speech text as a key and identify a comment that is found in the search (in other words, a comment including the keyword) as a comment that the streamer reacted.

The display controller145controls display in a viewer terminal300with respect to a comment that the above-described identifier144identified. For example, the display controller145, by sending command information to change a display form of the comment that the identifier144identified to a viewer terminal300, differentiates the display form of the identified comment (in other words, a comment that the streamer reacted) in the comment area of the viewer terminal300.

More specifically, a case where the above-described identifier144identifies the comment “How long is the driving time?” is described. In this case, the display controller145, for example, sends command information to change the character color and the background color of the comment to a viewer terminal300. In the viewer terminal300that has received the command information, for example, the display form of a comment DC (“How long is the driving time?”) is changed in a comment area CA, as illustrated inFIG.5. Note that change in a display form is not limited to change in the character color and the background color as described above, and, in addition thereto, may be change in a font, character size, or the like

The above-described CPU401or the like can function as the controller140having a configuration as described above.

(Functional Configuration of Streamer Terminal200)

FIG.6is a block diagram illustrating an example of a functional configuration of each streamer terminal200. As illustrated, the streamer terminal200includes an image capturer210, a voice input device220, a sender/receiver230, an operation acceptor240, a storage250, a controller260, and a display270.

The image capturer210is, for example, a camera arranged in the streamer terminal200and captures a video of the streamer and the like during distribution. In other words, the image capturer210outputs an image including the streamer during distribution. Note that the afore-described image capturing unit409can function as the image capturer210as described above.

The voice input device220input a voice of the streamer. For example, the voice input device220inputs a voice that the streamer utters during distribution. Note that the afore-described voice unit410can function as the voice input device220as described above.

The sender/receiver230sends and receives various types of information to and from the distribution server100via the Internet900. For example, the sender/receiver230, controlled by the controller260(a distributor262, to be described later), sends an image generated by synthesizing an object with an image captured by the image capturer210(more specifically, a video including voice input by the voice input device220) to the distribution server100(distributes the image to a viewer terminal300via the distribution server100). The sender/receiver230also receives a comment sent from the distribution server100(a comment posted from a viewer terminal300). Note that the afore-described NIC404can function as the sender/receiver230as described above.

The operation acceptor240is a touch panel or a pointing device and accepts various types of operations from the streamer. For example, the operation acceptor240accepts an operation on an object to be synthesized with an image captured by the image capturer210. Note that the afore-described operation unit411can function as the operation acceptor240as described above.

The storage250stores various types of information required for processing in the streamer terminal200. For example, the storage250stores information on an object to be synthesized with a captured image, a synthesis position of the object, and the like. As an example, the object is subtitles (banner) including textual information to be notified to a viewer in a live video. In addition, the synthesis position is a position at which an object is synthesized within an image. Note that the afore-described auxiliary storage407or the like can function as the storage250as described above.

The controller260controls the entire streamer terminal200. The controller260includes, for example, a synthesizer261and a distributor262.

The synthesizer261synthesizes an object within an image captured by the image capturer210. For example, the synthesizer261synthesizes textual information to be notified to a viewer within a captured image in accordance with information of the object and a synthesis position, which are stored in the storage250.

The distributor262sends an image with which an object is synthesized by the synthesizer261to the distribution server100for distribution to a viewer terminal300. That is, the distributor262controls the sender/receiver230to send an image with which an object is synthesized to the distribution server100and distributes the image to a viewer terminal300via the distribution server100.

Note that the afore-described CPU401or the like can function as the controller260having a configuration as described above.

The display270displays an image with which an object is synthesized by the above-described synthesizer261. In addition, the display270, as with a display350, to be described later, of a viewer terminal300, has a comment area and displays a comment received by the sender/receiver230. Note that the afore-described display unit412can function as the display270as described above.

(Functional Configuration of Viewer Terminal300)

FIG.7is a block diagram illustrating an example of a functional configuration of each viewer terminal300. As illustrated, the viewer terminal300includes a sender/receiver310, an operation acceptor320, a storage330, a controller340, and a display350.

The sender/receiver310sends and receives various types of information to and from the distribution server100via the Internet900. For example, the sender/receiver310, controlled by the controller340, receives a live video distributed from the distribution server100. In addition, the sender/receiver310sends a comment accepted through the operation acceptor320to the distribution server100and receives a comment and command information (command information to change the display form of a comment) sent from the distribution server100. Note that the afore-described NIC404can function as the sender/receiver310as described above.

The operation acceptor320is a touch panel or a pointing device and accepts various types of operations from the viewer. For example, the operation acceptor320accepts a comment input by the viewer. Note that the afore-described operation unit411can function as the operation acceptor320as described above.

The storage330stores various types of information required for processing in the viewer terminal300. For example, the storage330stores various types of applications including a video viewing app and a browser app. More specifically, various types of applications are installed in the storage330. Note that the video viewing app is, for example, an application to view a video distributed from the distribution server100. In addition, a video distributed from the distribution server100may also be able to be viewed through the browser app. Note that the afore-described auxiliary storage407or the like can function as the storage330as described above.

The controller340controls the entire viewer terminal300. For example, the controller340executes a video viewing app stored in the storage330and displays a live video or the like distributed from the distribution server100on the display350. Since, as described later, a video area for displaying a live video and a comment area for displaying a comment are defined on the display350, the controller340displays a live video that the sender/receiver310receives in the video area and displays a comment that the sender/receiver310receives in the comment area. The comment area is configured such that latest comments can be displayed in a predetermined number. Hence, while the number of comments does not reach the predetermined number, every time the sender/receiver310receives a comment, the controller340successively adds the received comment to the comment area. After the number of comments has reached the predetermined number, every time the sender/receiver310receives a comment, the controller340deletes an older comment from the comment area in a last-in-first-out manner and adds the new comment to the comment area. In addition, when the sender/receiver310receives command information, the controller340changes the display form of a comment targeted by the command information in the comment area in accordance with the command information. Note that the afore-described CPU401or the like can function as the controller340having a configuration as described above.

The display350displays a screen matching an application that the above-described controller340executes. For example, when the video viewing app is executed by the controller340, the display350displays a viewing screen VG1as illustrated inFIG.8. In the viewing screen VG1, a video area VA for displaying a live video and a comment area CA for displaying a predetermined number of posted comments are included. Note that, in the comment area CA, a comment DC to which the streamer reacted is displayed in a display form different from those of the other comments.

(Operation of Distribution Server100)

Operation of the distribution server100is described below with reference toFIG.9.FIG.9is a flowchart for a description of distribution processing according to Embodiment 1. The distribution processing is executed by the distribution server100when, for example, provision of a live video from a streamer terminal200is started.

First, the distribution server100starts distribution of a live video (step S11). That is, when the receiver120receives a live video provided by a streamer terminal200, the controller140(distribution controller141) controls the sender110to distribute the received live video to a viewer terminal300.

The distribution server100determines whether or not the distribution server100has received a new comment (step S12). That is, the controller140(comment manager142) determines whether or not the receiver120has received a comment posted from a viewer terminal300.

When the distribution server100determines that the distribution server100has not received a new comment (step S12; No), the distribution server100advances the process to step S14, to be described later.

In contrast, when the distribution server100determines that the distribution server100has received a new comment (step S12; Yes), the distribution server100updates comment management information and sends the received comment to respective terminals (step S13). That is, the controller140(comment manager142) updates the comment management information131, illustrated in the above-describedFIG.4. Specifically, the comment manager142adds the new comment to the comment management information131(sets “1” and “0” in the display flag131dand the reaction flag131eof the comment, respectively) and also changes the display flag131dof, among comments the display flags131dof which are “1”, a comment the reception time131bof which is the oldest to “0”. The controller140sends the new comment to the streamer terminal200and each viewer terminal300.

The distribution server100determines whether or not the streamer has spoken (step S14). That is, the controller140(speech recognizer143) determines whether or not the streamer has spoken in the live video.

When the distribution server100determines that the streamer has not spoken (step S14; No), the distribution server100advances the process to step S19, to be described later.

In contrast, when the distribution server100determines that the streamer has spoken (step S14; Yes), the distribution server100subjects speech of the streamer to speech recognition and converts the speech to a speech text (step S15). That is, the controller140(speech recognizer143), for example, segments voice data that the streamer has spoken into respective phoneme units, identifies respective phonemes through comparison with feature distributions of sample phonemes, subjects the identified phonemes to pattern matching with words, and the like, and subsequently finally generates a speech text representing speech that the streamer spoke.

The distribution server100compares the speech text with respective comments to be displayed (step S16). That is, the controller140(identifier144) compares the speech text converted in the above-described step S15with respective comments that are currently displayed in the comment area of a viewer terminal300. More specifically, the controller140first subjects the speech text to morphological analysis and extracts a keyword (for example, a noun or a noun phrase). Next, the controller140, referring to the comment management information131illustrated inFIG.4, subjects each comment the display flag131dof which is “1” to morphological analysis and extracts keywords of the respective comments. The controller140compares the keyword extracted from the speech text with the respective keywords extracted from the respective comments.

The distribution server100determines whether or not there is a coincident comment (step S17). For example, the controller140(identifier144) determines whether or not there is a comment that includes a keyword coinciding with the keyword included in the speech text through the comparison in the above-described step S16.

When the distribution server100determines that there is no coincident comment (step S17; No), the distribution server100advances the process to step S19, to be described later.

In contrast, when the distribution server100determines that there is a coincident comment (step S17; Yes), the distribution server100commands each terminal to change the display form of the coincident comment (step S18). That is, the controller140(display controller145), by sending command information to change the display form of the coincident comment to each viewer terminal300and the like, differentiates the display form of the comment to which the streamer reacted in the comment area of each viewer terminal300. For example, a viewer terminal300that has received the command information changes the display form of the comment DC (“How long is the driving time?”) and displays the comment DC in the comment area CA in the viewing screen VG1illustrated in the above-describedFIG.8.

The distribution server100determines whether or not the distribution of the live video is finished (step S19). That is, the controller140(distribution controller141) determines whether or not the live video provided by the streamer terminal200is finished.

When the distribution server100determines that the distribution of the live video has not been finished (step S19; No), the distribution server100returns the process to the above-described step S12.

In contrast, when the distribution server100determines that the distribution of the live video is finished (step S19; Yes), the distribution server100terminates the distribution processing.

The distribution processing as described above enables, among comments posted by viewers, a comment to which the streamer reacted to be displayed in a distinguishable manner. Furthermore, since, when a comment to which the streamer reacted is identified, only comments that are displayed in the comment area are to be compared with the speech text, it is possible to not only increase the precision of matching of a comment with the speech text but also perform processing of matching at high speed. As a result, it is possible to determine whether or not the streamer reacted to a posted comment with high precision and a low calculation load.

Although, in the above-described Embodiment 1, a case where the display form of a comment to which the streamer reacted is changed was described, the display of a comment may be controlled from other viewpoints. For example, the display controller145may cause a comment that the identifier144identified (comment to which the streamer reacted) to be displayed in the comment area for a longer period of time than a standard period.

When described in more detail, the display controller145, by sending command information to defer (renew) the reception time of the comment that the identifier144identified from the original reception time by a certain amount of time to the viewer terminal300, causes the identified comment to be displayed for a longer period of time in the comment area of a viewer terminal300. The display controller145may also, by including, in the command information, first command information to change the display form of a comment and second command information to defer the reception time of the comment from the original reception time by a certain amount of time, cause the comment the display form of which is changed to be displayed in the comment area for a longer period of time than a standard period. In such cases, it is possible to preferentially display a comment to which the streamer reacted.

Embodiment 2

Although, in the above-described Embodiment 1, a case where, when a comment to which the streamer reacted is identified, the display form of the comment is changed was described, a comment to which a streamer reacted may be used for setting a chapter. A distribution system characterized by using a comment to which a streamer reacted for setting a chapter is described below.

FIG.10is a schematic diagram illustrating an example of an overall configuration of a distribution system2according to Embodiment 2 of the present disclosure. In the distribution system2, a distribution server500that distributes a live video, a streamer terminal200that is used by a streamer, and a viewer terminal300that is used by a viewer are connected to one another in a communicable manner via the Internet900, as an example. Note that the streamer terminal200and the viewer terminal300are respectively the same constituent elements as those in the distribution system1according to the above-described Embodiment 1. In other words, only the distribution server500differs from that in the distribution system1inFIG.1.

The distribution server500is also achieved by an information processing device400in the above-describedFIG.2. That is, when the information processing device400is powered on, a program that cause the information processing device400to function as the distribution server500according to Embodiment 2 is executed, and the distribution server500according to Embodiment 2 is thereby achieved.

(Functional Configuration of Distribution Server500)

FIG.11is a block diagram illustrating an example of a functional configuration of the distribution server500according to Embodiment 2. As illustrated, the distribution server500includes a sender110, a receiver120, a storage530, and a controller540. Note that the sender110and the receiver120have the same configurations as the sender110and the receiver120in the distribution server100in the above-describedFIG.3, respectively.

The storage530stores various types of information required for processing in the distribution server500. For example, the storage530stores comment management information131and chapter management information531. Note that the comment management information131is the same information as the comment management information131in the above-describedFIG.4. The storage530also saves (records) a live video in parallel with distribution of the live video. In other words, the controller540(distribution controller141), while distributing a live video, saves the live video in the storage530. Because of this configuration, as described later, when a viewer who has belatedly started viewing a live video initiates so-called chasing playback in order to view the video from a scene that the viewer missed, the controller540is capable of, even during distribution of the live video, performing distribution using a part of the live video that has been saved in the storage530. In addition, when, after distribution of a live video was finished, a viewer initiates on-demand playback of the live video (past video), the controller540is capable of performing distribution using the live video saved in the storage530.

The chapter management information531is information to manage chapters into which a live video is divided and includes, as an example, information as illustrated inFIG.12. Specifically, in the chapter management information531, pieces of information, such as a chapter number531a, a relative time531b, a keyword531c, and an address531d, are included. Note that the relative time531bis time information indicating a start position of a chapter in a live video and is indicated by a relative time based on the start time point of the live video (elapsed time from the beginning of the video). The keyword531cis a phrase to identify a chapter and is also a phrase that is referred to by a streamer in the chapter. The address531dis a uniform resource locator (URL) to view a live video by specifying the start position of a chapter. Note that “?t=103” appended to the end of the URL is an example of an argument and defines that viewing is to be started from a position to which the video has advanced from the beginning of the live video when 103 seconds (a value indicated in seconds to which the value of the relative time531bis converted) have elapsed.

The storage530stores, in addition to the chapter management information531, various types of information required for processing performed by the controller540. For example, the storage530stores an ID, a password, and the like of a viewer who uses a viewer terminal300. The storage530also stores information relating to a live video. Note that the afore-described auxiliary storage407or the like can function as the storage530as described above.

Returning toFIG.11, the controller540controls the entire distribution server500. The controller540includes, for example, a distribution controller141, a comment manager142, a speech recognizer143, an identifier144, and a setter541. Note that the comment manager142, the speech recognizer143, and the identifier144have the same configurations as the comment manager142, the speech recognizer143, and the identifier144in the distribution server100(controller140) in the above-describedFIG.3, respectively. In addition, the distribution controller141further has, in addition to the functions of the distribution controller141of the above-described distribution server100, the following functions.

The distribution controller141, while distributing a live video to a viewer terminal300, saves the live video in the storage530. In addition, when, after the setter541, to be described below, has set a chapter in a live video, a viewer initiates chasing playback or on-demand playback of the live video, the distribution controller141displays the chapter in a seek bar, which indicates a playback position of the live video. Because of this configuration, by, in addition to performing operation on chapter information (chapter information CD displayed in a summary area SA), to be described later, moving the seek bar to a chapter, the viewer can view the live video from the start position of the chapter.

The setter541sets a chapter to divide a live video, based on a relative time in the live video (in other words, a relative time based on the start time of the live video) corresponding to a time at which a comment that the identifier144identifies is received. For example, the setter541sets a chapter by generating the chapter management information531as illustrated in the above-describedFIG.12and adding the generated chapter management information531to summary information of the live video.

More specifically, a case where the identifier144has extracted a keyword, such as “driving time”, from a speech text and has identified a comment “How long is the driving time?” from comments that are currently displayed in a comment area is described as an example. Note that it is assumed that the start time of the live video is “20:20:00” and the address of the live video is “https://XXXX.com/AbCdleFgH2IjK”. In this case, the setter541first calculates a relative time in the live video by subtracting the start time of the live video from a reception time131bof the corresponding comment (the comment “How long is the driving time?” in the column of a comment131c) in the comment management information131illustrated inFIG.4. Specifically, when the setter541obtains “0:01:43” by subtracting “20:20:00” from “20:21:43”, the setter541sets “0:01:43” in the relative time531bin the chapter management information531illustrated inFIG.12. Next, the setter541sets a keyword that the identifier144used at the time of identifying the comment (in other words, the keyword “driving time”) in the keyword531cin the chapter management information531. Finally, the setter541appends the argument “?t=103” (viewing is started from a position to which the video has advanced from the beginning of the video when 103 seconds has elapsed) to the address of the live video and sets the resultant address in the address531din the chapter management information531.

The setter541sets the chapter by adding the chapter management information531generated in this way to the summary information on the live video. For example, when, after the chapter management information531as illustrated inFIG.12is added to the summary information on the live video, the live video is viewed by a viewer terminal300, a display350of the viewer terminal300displays a viewing screen VG2as illustrated inFIG.13. In the viewing screen VG2, a video area VA for displaying a live video, the summary area SA for displaying a summary of the live video, and a comment area CA for displaying a predetermined number of posted comments are included. Note that, to the summary area SA, chapter information CD is added. The chapter information CD is linked to the address531din the chapter management information531, and a viewer can, by performing touch operation on the chapter information CD, view the live video from the start position of the chapter.

(Operation of Distribution Server500)

Operation of the distribution server500is described below with reference toFIG.14.FIG.14is a flowchart for a description of distribution processing according to Embodiment 2. The distribution processing is executed by the distribution server500when, for example, provision of a live video from a streamer terminal200is started. Note that, in steps S11to S17and S19in the distribution processing inFIG.14, the same processing as the processing in steps S11to S17and S19in the distribution processing in the afore-describedFIG.9is performed, respectively. Therefore, the same processing is simply described.

First, the distribution server500starts distribution of a live video (step S11) and determines whether or not the distribution server500has received a new comment (step S12).

When the distribution server500determines that the distribution server500has not received a new comment (step S12; No), the distribution server500advances the process to step S14, to be described later. In contrast, when the distribution server500determines that the distribution server500has received a new comment (step S12; Yes), the distribution server500updates comment management information and sends the received comment to respective terminals (step S13).

The distribution server500determines whether or not the streamer has spoken (step S14).

When the distribution server500determines that the streamer has not spoken (step S14; No), the distribution server500advances the process to step S19, to be described later. In contrast, when the distribution server500determines that the streamer has spoken (step S14; Yes), the distribution server500subjects speech of the streamer to speech recognition and converts the speech to a speech text (step S15).

The distribution server500compares the speech text with respective comments to be displayed (step S16) and determines whether or not there is a coincident comment (step S17).

When the distribution server500determines that there is no coincident comment (step S17; No), the distribution server500advances the process to step S19, to be described later.

In contrast, when the distribution server500determines that there is a coincident comment (step S17; Yes), the distribution server500sets a chapter to divide the live video (step S21). That is, the controller540(setter541) sets a chapter to divide the live video, based on a relative time in the live video (in other words, a relative time based on the start time of the live video) corresponding to a time at which the coincident comment is received. For example, the controller540sets a chapter by generating the chapter management information531as illustrated in the afore-describedFIG.12and adding the generated chapter management information531to the summary information of the live video.

The distribution server500determines whether or not the distribution of the live video is finished (step S19).

When the distribution server500determines that the distribution of the live video has not been finished (step S19; No), the distribution server500returns the process to the above-described step S12. In contrast, when the distribution server500determines that the distribution of the live video is finished (step S19; Yes), the distribution server500terminates the distribution processing.

The distribution processing as described above enables a comment to which the streamer reacted to be identified and a chapter to divide the live video to be easily set, based on a relative time in the live video corresponding to a reception time of the identified comment. In other words, since not a time at which the streamer reacted but a time at which a comment to which the streamer reacted was received is used, a chapter is set at a position that allows a topic having been talked about before the streamer reacted to the comment to be understood. As a result, it is possible to appropriately set a chapter to divide a live video. In addition, although, conventionally, chapter setting has been a heavy burden because a streamer or a viewer manually set a chapter, it is possible to eliminate such a human burden because the present embodiment enables a chapter to be automatically set.

Although, in the above-described Embodiment 2, a case where, based on generated chapter management information, chapter information CD is added to the summary area SA as illustrated inFIG.13was described, it may be configured such that a message including such chapter information CD can be posted to a social networking service (SNS). A user who, after a message including chapter information CD has been posted to an SNS, has viewed the message performing operation on the chapter information CD causes, for example, a video viewing app to be started and enables the live video to be viewed from the start position of the chapter in the same manner as described above.

Other Embodiments

Although, in the above-described Embodiments 1 and 2, a case where, by comparing a speech text, which is generated by subjecting speech spoken by a streamer in a live video to speech recognition, with respective comments currently being displayed, a comment to which the streamer reacted is identified was described, a comment to which a streamer reacted may be identified using other methods. For example, the streamer terminal200may be configured to allow the streamer to arbitrarily select a posted comment, and a comment selected by the streamer may be identified as a comment to which the streamer reacted.

When described in more detail, the display270of the streamer terminal200, as with the display350of the viewer terminal300, has a comment area and displays a comment (posted comment) received by the sender/receiver230in the comment area. When the streamer who is distributing the live video selects one of comments displayed in the comment area, the controller260sends selection information indicating a selected comment to the distribution server100or500through the sender/receiver230. In the distribution server100or500that has received the selection information, the identifier144, based on the selection information, identifies a comment to which the streamer reacted. In this case, it is possible to identify a comment to which the streamer reacted without error.

Although, in the above-described Embodiments 1 and 2, cases where the speech recognizer143, the identifier144, and the like are included in the controllers140and540of the distribution servers100and500and the distribution servers100and500identify a comment (comment to which the streamer reacted) were described, respectively, the configurations of the speech recognizer143, the identifier144, and the like may be added to the controller260of the streamer terminal200and the streamer terminal200may thereby identify a comment.

In the above-described Embodiments 1 and 2, programs that are executed by the distribution servers100and500and the like can be stored in a computer-readable recording medium, such as a compact disc read only memory (CD-ROM), a DVD, a magneto-optical disk (MO), a USB memory, and a memory card, and distributed. By installing such programs in a specific or general-purpose computer, it is possible to cause the computer to function as the distribution servers100and500in the above-described Embodiments 1 and 2, respectively.

In addition, it may be configured such that the above-described programs are stored in a disk device that a server device on a communication network, such as the Internet, has and are, for example, superimposed on a carrier wave and downloaded to a computer. In addition, by starting and executing programs while transferring the programs via a communication network, the above-described processing can be achieved. Further, by causing a server device to execute all or some of programs and a computer to execute a program while sending and receiving information relating to the processing in the server device via a communication network, the above-described processing can also be achieved.

Note that, when the above-described functions are achieved by the operating system (OS) sharing some of the functions, are achieved by collaboration between the OS and applications, or the like, only programs other than the OS may be stored in the above-described recording medium and distributed or may be downloaded to a computer.

A server device, including:one or more processors, wherein at least one of the one or more processors executes processing of:distributing a live video provided from a streamer to a viewer terminal having a comment area, the comment area allowing a predetermined number of comments to be displayed;receiving a comment posted from the viewer terminal during distribution of the live video;sending the received comment to the viewer terminal to cause the comment to be added to the comment area in a last-in-first-out manner;identifying a comment to which the streamer reacted in the live video being distributed by, every time the streamer speaks, comparing a speech text, the speech text being acquired by subjecting speech of the streamer to speech recognition, with respective comments currently being displayed in the comment area; andcontrolling display of the identified comment.

The server device according to [1], wherein the control of display is control to cause a display form of the identified comment to be changed in such a manner as to be differentiated from a display form of another comment.

The server device according to [1] or [2], wherein the control of display is control to cause the identified comment to be displayed in the comment area for a longer period of time than a standard period.

The server device according to any one of [1] to [3], wherein, in the identification of a comment, the server device compares a keyword included in the speech text with a keyword included in each of the comments and identifies a comment including a keyword coinciding with the keyword in the speech text as a comment to which the streamer reacted.

A display control method, includingexecuting, by a computer,distributing of a live video provided from a streamer to a viewer terminal having a comment area, the comment area allowing a predetermined number of comments to be displayed;receiving of a comment posted from the viewer terminal during distribution of the live video;sending of the received comment to the viewer terminal to cause the comment to be added to the comment area in a last-in-first-out manner;identifying of a comment to which the streamer reacted in the live video being distributed by, every time the streamer speaks, comparing a speech text, the speech text being acquired by subjecting speech of the streamer to speech recognition, with respective comments currently being displayed in the comment area; andcontrolling of display of the identified comment.

A non-transitory computer-readable recording medium storing a program, the program causing a computer to execute processing of:distributing a live video provided from a streamer to a viewer terminal having a comment area, the comment area allowing a predetermined number of comments to be displayed;receiving a comment posted from the viewer terminal during distribution of the live video;sending the received comment to the viewer terminal to cause the comment to be added to the comment area in a last-in-first-out manner;identifying a comment to which the streamer reacted in the live video being distributed by, every time the streamer speaks, comparing a speech text, the speech text being acquired by subjecting speech of the streamer to speech recognition, with respective comments currently being displayed in the comment area; andcontrolling display of the identified comment.

INDUSTRIAL APPLICABILITY

The present disclosure is suitably applicable to video distribution in which display of comments posted during distribution is controlled.

REFERENCE SIGNS LIST

1,2Distribution system100,500Distribution server110Sender120Receiver130,530Storage131Comment management information531Chapter management information140,540Controller141Distribution controller142Comment manager143Speech recognizer144Identifier145Display controller541Setter200Streamer terminal210Image capturer220Voice input device230Sender/receiver240Operation acceptor250Storage260Controller261Synthesizer262Distributor270Display300Viewer terminal310Sender/receiver320Operation acceptor330Storage340Controller350Display400Information processing device401CPU402ROM403RAM404NIC405Image processor406Sound processor407Auxiliary storage408Interface409Image capturing unit410Voice unit411Operation unit412Display unit900Internet