PROGRAM, TERMINAL DEVICE, AND METHOD

The terminal device 10 of a screen presentation system 1 comprises: a connection detection unit 152 for detecting the connection of the terminal device to a display device; a photographing control unit 153 for controlling the photographing of a viewer by a camera 130; a viewer state information acquisition unit 155 for detecting the state of the viewer and acquiring as information that indicates the state of the viewer; and a video display control unit 157 for presenting the video to the display device.

BACKGROUND ART

Conventionally, the distribution of video content via the communication networks such as the internet has been performed by several video distributors.

Patent document 1 discloses a technology for distributing digital video, interactive data such as internet data, and conventional telephone services (POTS). This technology provides multiple video channels and interactive data via communication lines, and manages access privileges and account status to prevent unauthorized use.Patent Document 1: JP-A-2003-521130

SUMMARY

Technical Problem

By the way, when streaming video content, it is important to prevent unauthorized use and ensure collection of viewing fees. It is also important to make the system easy to use, without requiring complicated setup work. As described above, technology for a video distribution system that prevents unauthorized use and enables easy use has been desired.

Thus, this disclosure describes a technology for a video distribution system that prevents unauthorized use and is easy to use.

Solution to Problem

According to one embodiment of the disclosure, a program for execution by a computer provided with a processor and connected to a display device displaying video, wherein the program is configured to cause the processor to execute a step of detecting the connection of the terminal device to the display device; a step of capturing a viewer viewing the video displayed on the display device and generating the captured image information; a step of acquiring information indicating a state of the viewer based on the captured image information; and a step of receiving image information related to the image delivered from an external source based on the information indicating the state of the viewer, and displaying the image on the said display device.

According to another embodiment of the disclosure, a terminal device connected to a display device displaying video comprising a control unit and a photographing unit; wherein the photographing unit captures an image of a viewer viewing the video displayed on the display device and generates captured image information; and wherein the control unit executes a step of detecting the connection of the terminal device to the display device; a step of acquiring information indicating a state of the viewer based on the captured image information; and a step of receiving video information distributed from an external source and displaying the video on the display device based on the state of the viewer.

According to another embodiment of the disclosure, a method for execution by a computer provided with a processor and connected to a display device displaying video, wherein the method comprises a processor executing a step of detecting the connection of the terminal device to the display device; a step of capturing a viewer viewing the video displayed on the display device and generating the captured image information; a step of acquiring information indicating a state of the viewer based on the captured image information; and a step of receiving video information distributed from an external source and displaying the video on the display device on the state of the viewer.

According to the present disclosure, when the system detects that the terminal device is connected to a display device, the terminal device takes a picture of the viewer viewing the video, acquires information indicating the viewer's status, and the system displays the video on the display device. This prevents unauthorized use and makes it possible to construct a video distribution system which is easy to use.

DETAILED DESCRIPTION

The following is a description of the Embodiment of the present disclosure with reference to the drawings. In the following description, identical components are marked with the same symbol. Their names and functions are also the same. Therefore, a detailed explanation of them is not repeated.

Overview

The following describes a screen presentation system of the present disclosure. The screen presentation system is a system for screening (displaying) various types of images, such as film contents, broadcast images of sports events, concerts, talk events, etc., on display devices such as television receivers (including smart TVs with information processing functions), displays and projectors installed in private homes, shops, offices, etc. This is a system for screening (displaying) various types of images, such as movie-like image content, live broadcast images of sports events, concerts, talk events, etc., and classroom images for educational use. The screening (display) of various types of video can be either real-time filmed and distributed (so-called live relay) or recorded. Furthermore, the images may be available for viewing only at predetermined times, or at arbitrary times.

In this screen presentation system, for example, the program contents, distribution dates and times, viewing fees, etc., of the various videos described above are indicated on a web page provided by a company operating the screen presentation system. Users (viewers) who wish to view these various videos access the web pages provided by the operators of the screen presentation system from their own mobile terminals such as smartphones and tablets or PCs (Personal Computer), select the video they wish to view, and make an application to view the video. At this time, the user purchases the right to view the video in the form of a ticket using various payment methods (e.g., cashless payment by credit card, prepaid card, debit card, etc., or payment by various means of remittance). At the predetermined delivery date and time, at the server of the operator, the user is authenticated as the legitimate right holder, i.e., the person who purchased the ticket, by means of a predetermined authentication, and the distribution of the video is started.

In order to provide such screen presentation system, a dongle-type terminal device is provided in this screen presentation system. This terminal device is sent and rented (or may be transferred) by the operator when the user applies to view the images and purchases the ticket. When the terminal device it connected to a display device such as a television receiver or display, the terminal device communicates with the operator's server to receive the various types of images and displays on the display device. Therefore, this terminal device is therefore provided with connection ports to display devices, such as a USB (Universal Serial Bus) port, an HDMI (High-Definition Multimedia Interface) (registered trademark) port, a VGA (Video Graphics Array) port, and Video Graphics Array) port.

The terminal device is not necessarily be sent each time a user makes an application for video viewing. The terminal device sent once to the user may be used repeatedly. According to this configuration, the user only required to connect the dongle-type terminal device to the display device to view the video.

The terminal device is also provided with a camera. When the terminal device is connected to the display unit, the camera filming the surroundings the user (usually a viewer in front of the display unit). This screening system authenticates that the user is legitimate. The authentication is based on determining, in the captured image information taken by the camera, whether the user in the camera is a user with legitimate rights who has purchased a ticket to view the video, and whether the number of users specified when the user purchased a ticket (whether there are more than the specified number of users in the image) is in the image. The terminal device then starts showing (displaying) the video only when the authentication use is completed. This configuration makes it possible to distribute images to legitimate users.

The screen presentation system1is described below. In the following description, for example, when a terminal device10accesses a server20, the server20responds with information to the terminal device10. Also, when a user terminals30accesses the server20, the server20responds with information for generating a screen at the user terminals30. The user terminals30generates and displays a screen based on the information received from the server20.

<1. Overall Configuration of Screen Presentation System1>

FIG.1shows the overall structure of the screen presentation system1. As shown inFIG.1, the screen presentation system1includes the terminal device10, the server20and a plurality of user terminals (inFIG.1, user terminal30A and user terminal30B are shown. Hereinafter collectively referred to as “user terminal30”). The terminal device10, the server20and the user terminals30are mutually communicably connected via the network80. The network80comprises a wired or wireless network.

The terminal device10is a dongle-type terminal device for providing screening of video images. A user is, for example, a person who wishes to view or is viewing various images by the screen presentation system1, and who uses a display device such as a television receiver or a display. The appearance and use of the terminal device10is described below.

The terminal device10is communicatively connected to the server20via the network80. The terminal device10is connected to the network80by communicating with communication devices such as a wireless base station81compatible with communication standards such as 4G, 5G and LTE (Long Term Evolution), a wireless LAN (Local Area) such as IEEE (Institute of Electrical and Electronics Engineers) 802.11 Network) standards, such as IEEE (Institute of Electrical and Electronics Engineers) 802.11, and a wireless LAN (Local Area Network) router82, which is connected to the network80.

The server20is a device that manages user information and video distribution and is managed by the operator of the screen presentation system1. The server20accepts applications for video viewing through operations by users from user terminal30. The server20also performs the prescribed authentication for the user and transmits the video information to allow legitimate users to view the video.

The server20is a computer connected to network80. The server20has a communication IF (interface)22, an input/output IF23, a memory25, a storage26and a processor29.

The communication IF22is an interface for the server20to communicate with external devices and to input and output signals. The input/output IF23functions as an interface with input devices for receiving input operations from the operator (operator) and output devices for presenting information to the operator. The memory25is for temporarily storing programs, and data and other data processed by the programs, and is a volatile memory such as DRAM (Dynamic Random Access Memory), for example. The storage26is a storage device for storing data, e.g., flash memory, HDD (Hard Disc Drive). The processor29is hardware for executing the instruction set described in the program and comprises arithmetic units, registers, peripheral circuits.

The user terminals30are a device operated by each user. As shown inFIG.1as user terminal30A, user terminals30is a smartphone, tablet, or other terminal device compatible with the mobile communication system. In addition, the user terminal30may also be a stationary PC (Personal Computer) or a laptop PC, for example.

The user terminal30is communicatively connected to the server20via the network80. The user terminals30are connected to the network80by communicating with communication devices such as a wireless base station81compatible with communication standards such as 4G, 5G and LTE, and a wireless LAN router82compatible with wireless LAN standards such as IEEE 802.11. As shown inFIG.1as user terminals30B, user terminal is equipped with a communication IF32, an input device33, an output device34, a memory a storage unit36and a processor39.

The communication IF32is an interface for inputting and outputting signals in order for the user terminal30to communicate with external devices. The input device33is the device (e.g., touch panel, touch pad, pointing device such as a mouse, keyboard, etc.) for receiving input operations from the user. The output device34is the device (e.g., display, speaker, etc.) for presenting information to the user. Memory35is for temporarily storing programs and data and the like processed by the programs and the like, e.g., volatile memory such as DRAM. Memory36is a storage device for storing data, e.g., flash memory, HDD (Hard Disc Drive). Processor39is hardware for executing the instruction set described in the program and comprises arithmetic units, registers, peripheral circuits, etc.

<1.1. Configuration of the Terminal Device10>

FIG.2shows a diagrammatic view of the terminal device10comprising the screen presentation system1of the Embodiment 1. As shown inFIG.2, the terminal device10comprises a body11, a window12, a support13and a connector14.

The body11is a housing for storing various devices for realizing the functions provided by the terminal device10. The shape of the body11is not limited to an oval body as shown inFIG.2but can be any shape that can store various devices. The shape of the body11may be large enough to store a camera130, as described below, and large enough to be connected to the TV40and not obstruct the surrounding area.

The window12is provided to enable the camera130, described below, to photograph the surroundings of the terminal device10. The support13is a protruding portion provided to prevent the terminal device10from crashing when connected to the TV40described below. The support13may be configured to be refractile by a material whose shape can be changed in order to support the body11stably when connected to the TV40. The connector14is a connection port for connecting to the TV40, which is described below, and comprises a USB port, an HDMI port, a VGA port or the like.

FIG.3shows a front view of the terminal device10comprising the screen presentation system1of the Embodiment 1 connected to a television40. As shown inFIG.3, the terminal device10is used connected to a television40, which is an example of a display device, to screen (display) various images on the television40. The television40shown inFIG.3is, for example, a liquid crystal television and is provided with an external connection terminal41for receiving video signals, etc. from external source. The external connection terminal41comprises a USB port, an HDMI port, a VGA port, etc.

As shown inFIG.3, the terminal device10receives video information transmitted from the server20by connecting the connector14to the external connection port41of the television40and shows (displays) various images as video on the television40. In addition, the camera130provided inside the body11photographs the surroundings through the window12. If the support13is efractile, it may be refracted so that the body11is directed upwards. The terminal device10may be configured to operate by receiving power from the television40or may be configured to operate by receiving power from a battery or an external power supply. In this Embodiment, the television40is shown as an example of a display device to which the terminal device10is connected, but it may be configured to be connected to a projector that shows images on an LCD panel, display, screen, or the like. It may also be configured to have more than one display device (TV, LCD panel or screen) displaying images and to display different images on each display device. In such a case, the terminal device10may be configured to connect to a control unit or the like that controls the display of images on the plurality of display devices.

FIG.4shows a block diagram of the functional configuration of the terminal device10comprising the screen presentation system1. As shown inFIG.4, the terminal device10comprises a plurality of antennas (an antenna111, an antenna112), wireless communication units corresponding to each antenna (first wireless communication unit121, second wireless communication unit122), the camera130, a storage unit140and a control unit150. The terminal device10also has functions and configurations not specifically illustrated inFIG.2(e.g., a battery for holding power, a power supply circuit for controlling the supply of power from the battery to each circuit, etc.). As shown inFIG.2, each block included in the terminal device10is electrically connected by a bus or other means.

The antenna111radiates signals emitted by the terminal device10as radio waves. The antenna111also receives radio waves from the space and gives the received signals to the first wireless communication unit121.

The antenna112radiates signals emitted by the terminal device10as radio waves. The antenna112also receives radio waves from the space and gives the received signals to the second wireless communication unit122.

The first wireless communication unit121performs modulation and demodulation processing, etc. for transmitting and receiving signals via the antenna111in order for the terminal device10to communicate with other wireless devices. The second wireless communication unit122performs modulation and demodulation processing, etc. for transmitting and receiving signals via the antenna112in order for the terminal device10to communicate with other wireless devices. The first wireless communication unit121and the second wireless communication unit122are communication modules including tuners, RSSI (Received Signal Strength Indicator) calculation circuits, CRC (Cyclic Redundancy Check) calculation circuits and high frequency circuits. The first wireless communication unit121and the second wireless communication unit122perform modulation and frequency conversion of the radio signals transmitted and received by the terminal device10and provide the received signals to the control unit150.

The camera130is a device for receiving light with a light receiving element and outputting it as a captured image. The camera130is, for example, a depth camera capable of detecting the distance from the camera130to the object being photographed.

The storage unit140comprises, for example, a flash memory and stores data and programs used by the terminal device10. In one aspect, the storage unit140stores equipment information141, viewer information142and video information143.

The equipment information141is information on the equipment of the terminal device10. The equipment information includes information that identifies the terminal device10registered on the server20(equipment ID), device ID, IP address, etc.

The viewer information142is information of a user (viewer) who applies to view various videos via the user terminal30. The viewer information includes information that identifies the viewer registered on the server20(viewer ID), the viewer's name, age and gender.

The video information143is video information delivered from the server20and temporarily stores the received video information.

The control unit150controls the operation of the terminal device10by reading a program stored in the storage unit140and executing the instructions contained in the program. The control unit150is, for example, an application that is incorporated or installed in the terminal device10in advance. According to the program, the control unit150performs as a transmission receiving unit151, a connection detection unit152, a photographing control unit153, a viewer identification information acquisition unit154, a viewer status information acquisition unit155, a viewer identification unit156and a video display control unit157.

The transmission receiving unit151controls the process by which the terminal device transmits and receives data with external devices, such as the server20, according to a communication protocol.

The connection detection unit152controls the process of detecting connection of the terminal device10to a display device such as a television receiver or the display. Specifically, when the connector14of the terminal device10is connected to the external connection port of the display device and becomes ready for communication, the connection detection unit152detects a signal transmitted from the display device or a response signal to a signal transmitted to the display device. Thereby, the connection detection unit152detects that the terminal device is connected to the display device and ready for communication.

The photographing control unit153controls the process of photographing the viewer by the camera130. For example, when the connection detection unit152detects the connection of the terminal device to the display device, the photographing control unit153activates the camera130, causes the camera130to take pictures of the surroundings of the display device and generates the photographed image information. Furthermore, the photographing control unit153moves the camera130in the photographing direction. When a person is detected in the photographing image, the control unit controls the camera to continue photographing the person as a viewer. A person may be detected by a human sensor (illustration omitted) provided by the terminal device10, and the camera130may be controlled to continue photographing by moving the photographing direction of the camera130in the direction of the detected person.

The viewer identification information acquisition unit154controls the process of the server20to transmit information that identifies the viewer who has applied for video viewing to the server20. The viewer identification information is, for example, identification information such as a code number or QR code (registered trademark) transmitted by the server20to the user terminal30when a viewer makes an application for video viewing at the user terminal30, to enable authentication that the viewer in question is a viewer who has legitimately applied for video viewing. This is information that enables the viewer to be authenticated as a viewer who has legitimately applied for video viewing.

The viewer identification information may be a facial photograph identifying the viewer's face. For example, when a viewer applies for video viewing at the user terminal30, the viewer identification information acquisition unit154acquires the facial photograph data of the viewer's own face taken at the user terminal30in accordance with instructions from the server20.

Furthermore, the viewer identification information may be information identifying the number of viewers, for example, information on the number of viewers (numerical information), or may be a photograph of the faces of all viewers. For example, the viewer identification information acquisition unit154acquires information on the number of persons identified by the viewer when the viewer makes an application for video viewing at the user terminal30, or the face photograph data of all viewers taken at the user terminal30in accordance with instructions from the server20.

The viewer state information acquisition unit155detects the viewer state from the image information of the viewer captured by the camera130and controls the process of acquiring the information indicating the viewer state. Information indicating the state of the viewer is, for example, information indicating the direction of the viewer's face when the viewer is in front of the display unit, facial expressions, the viewer's body posture, and whether the viewer is in an awake or asleep state. For example, the information indicating these viewer states is used to determine whether the viewer is in a state in which he or she can view the video.

The acquisition of information indicating the state of the viewer by the viewer state information acquisition unit155uses, for example, a known technique of face recognition. This face recognition is performed, for example, by matching an object in the captured image information with a database of face information held as a database not shown, and determining whether it is a face based on the positional relationship between each part, such as eyebrows, eyes, nose and mouth, and the contour of the object. This enables the orientation of the viewer's face to be determined.

The viewer identification unit156controls the process of identifying the viewer captured by the camera130as a legitimate viewer who has applied for video viewing to the server20. For example, if the viewer identification information acquisition unit154has acquired a code number or QR code as information identifying the viewer, the viewer identification unit156causes the code number or QR code to be presented to the viewer via the user terminal30so that the information can be captured by the camera130and read. If the information read matches the information obtained, the viewer is authenticated as a viewer who has legitimately applied for video viewing.

When the viewer identification information acquiring unit154has acquired the viewer's facial photograph as information identifying the viewer, the viewer identification unit156instructs the viewer via the user terminal30to face the front and detects the viewer's face from the captured image information generated by the photographing control unit153. If the acquired viewer's face photograph and the viewer's face detected from the captured image information match, the viewer is authenticated as a viewer who has legitimately applied for video viewing. At this time, the viewer identification unit156may, use the known technique of face recognition, or the viewer's status by the viewer status information acquisition unit155. Furthermore, the viewer identification unit156may, extract a feature value (first feature value) of the viewer's face in the acquired viewer's face photograph (viewer image information), extract a feature value (second feature value) of the viewer's face detected from the captured image information, and if the respective feature values match a predetermined value or a predetermined percentage or more, authenticate the viewer as the viewer.

Furthermore, if the viewer identification information acquiring unit154acquires information identifying the number of viewers as information identifying the viewers, the viewer identification unit156instructs the viewers (whether some or all) to face the front via the user terminals30and the shooting image generated by the photographing control unit153. If the number of viewers acquired is compared with the number of viewer faces detected from the captured image information and they match, the viewer identification unit156authenticates that all the viewers are legitimate viewers. Even if the number of viewers detected from the captured image information is less than the number of viewers acquired, it may still authenticate that all such viewers are legitimate viewers, because the number of viewers is at least not more than the number of viewers authorized to view the image.

The identification of the viewer by the viewer identification unit156may be performed by the server20. In this case, the captured image information may be transmitted to the server and the terminal device10may not have the viewer identification information acquisition unit154and the viewer identification unit156as functions of the terminal device10.

The video display control unit157receives video information transmitted from the server20and controls the process of showing (displaying) various images as video on the display device. For example, the video may be displayed only when the viewer identification unit156can identify that the viewer captured by the camera130is a legitimate viewer. The video may be displayed only when information indicating the viewer's status is determined by the viewer status information acquisition unit155to be in a state where the viewer is able to view the video.

For example, by equipping the terminal device10with a sound output device such as a speaker, the video display control unit157may be configured to receive the video information transmitted from the server20, cause images to be shown (displayed) on a display device and cause sound to be output from a sound output device. For example, by setting the sound output device of the terminal device10to a state suitable for viewing the video, it is possible to view the video with higher quality sound.

<1.2. Functional Configuration of the Server20>

FIG.5shows the functional configuration of the server20comprising the screen presentation system1of the Embodiment 1. As shown inFIG.5, the server20functions as a communication unit201, a storage unit202and a control unit203.

The communication unit201processes the server20to communicate with external devices.

The storage unit202stores data and programs used by the server20. The storage unit202stores a viewer database2021, an equipment database2022, a viewing purchase database2023, a video database2024, etc.

The viewer database2021is a database for holding information on users (viewers) who view various videos in the screen presentation system1. Details are described below.

The equipment database2022is a database for holding information on the equipment of the terminal device10used in the screen presentation system1. Details are described later.

The viewing purchase database2023is a database for holding information on the rights (tickets) to view images purchased by users (viewers) to view various images using the screen presentation system1. Details are described later.

The video database2024is a database for holding video information to be shown (displayed) in the screen presentation system1. This video information is information on the video to be displayed on a display device such as a television via the terminal device10. The server20does not need to hold the actual video information as the video database2024, for example, it may be managed by another server and hold only link information to that server.

The control unit203performs as the various modules as the program cause processor of the server20to execute processing. The various modules comprise a receiving control module2031, a transmission control module2032, a viewing acceptance module2033, a viewer identification information transmission module2034, a viewer authentication module2035and a video information distribution module2036.

The receiving control module2031controls the process by which the server20receives signals from external devices according to a communication protocol.

The transmission control module2032controls the process by which the server20transmits signals to external devices according to a communication protocol.

The viewing acceptance module2033controls the process of accepting the application for viewing the video when the viewer makes an application for viewing the video at the user terminal30. The viewing acceptance module2033, for example, transmits to the user terminal30selectable information such as the type of video available for viewing (e.g., the genre and category), the title of the video, the available viewing time and the ticket price, and presents this information to the viewer. The viewer operates the user terminals30and selects from among the presented images, and the viewing acceptance module2033receives the information on the selected image. The server20then processes the sending of the terminal device10to the viewer.

The viewer identification information transmission module2034controls the process of transmitting viewer identification information as a response to the transmission request from the viewer identification information acquisition unit154of the terminal device10, which transmits a transmission request for information identifying the viewer who has applied for video viewing. The viewer identification information includes, as described above, identification information such as a code number or QR code, a facial photograph identifying the viewer's face, and information identifying the number of viewers, such as headcount information and facial photographs of all multiple viewers.

The viewer authentication module2035controls the process of authenticating that the viewer filmed by the camera130of the terminal device10is a legitimate viewer who has applied to the server20for video viewing. The authentication performed by the viewer authentication module2035is the same as the authentication of the viewer performed by the viewer identification unit156of the terminal device10, and when such function is performed by the terminal device10, the viewer authentication module2035need not be provided as a function of the server20.

The video information distribution module2036controls the process of distributing video information to the terminal device10of a viewer who applied to view the video. The video information distribution module2036may be configured to deliver the video information to a specific viewer according to the viewer information stored in the viewer database2021. Specifically, the information may be delivered only to viewers whose attributes, such as viewer gender and age, are specific (e.g., women in their 20s, etc.), or only to viewers at a specific address (e.g., Tokyo, or Kansai region, etc.). Furthermore, the video information distribution module2036may be configured to distribute advertisement information (e.g., distributing advertisements for youth-oriented automobiles to males in their 20s to 30s, etc.) to specific viewers according to viewer information stored in the viewer database2021. The advertising information may be still images or video.

2. Data Structure

FIG.6shows the data structure of the viewer database2021, equipment database2022and viewing purchase database2023stored by the server20.

As shown inFIG.6, each of the records in the viewer database2021includes the item “viewer ID”, the item “viewer name”, the item “age”, the item “sex”, the item “address”, the item “face photo data”, the item “viewing status information”, the item “rental equipment ID”.

The item “viewer ID” is information that identifies each viewer who has applied for video viewing. The operation of the screen presentation system1may require membership registration to apply for video viewing, in which case the information may be used to identify each member.

The item “viewer name” is information indicating the name of the viewer who applied for video viewing.

The item “age” is information indicating the age of the viewer who applied for video viewing.

The item “sex” indicates the gender of the viewer who applied to view the video.

The item “address” indicates the address of the viewer who applied for video viewing.

The item “face photo data” is photo data that identifies the face of the viewer who made the application to view the video.

The item “viewing status information” is the analysis information of the viewer who has applied for video viewing, based on which the viewer's video viewing is analyzed. The analysis of viewers is described below.

The item “rental equipment ID” is information that identifies each of the10terminals loaned to viewers who have applied for video viewing.

Each of the records in the equipment database2022includes the item “equipment ID,” the item “device ID,” the item “IP address,” the item “communication history data,”.

The item “equipment ID” is information that identifies each of the terminal devices10lent to viewers who have applied for video viewing and corresponds to the item “device ID” in the viewer database2021.

The item “device ID” is information for identifying the terminal device10and is information for identifying individual devices assigned by the device manufacturer.

The item “IP address” is the address information of each of the terminal device10, which is necessary when communicating with the terminal device10via the internet.

The item “communication history data” is the history information of communication with the terminal device10and is used to determine whether communication with the terminal device10was successful or not and is used as maintenance information for the terminal device10.

Each of the records in the viewing purchase database2023includes the item “viewer ID,” the item “video ID,” the item “number of available viewers,” the item “viewer information,” the item “state information,”.

The item “viewer ID” identifies each viewer who has applied to view the video and corresponds to the item “viewer ID” in the viewer database2021.

The item “video ID” is information that identifies each of the image information pertaining to the image and corresponds to the item “video ID” that identifies each of the image information in the video database2024(illustration omitted).

The item “number of available viewers” is information that identifies the number of viewers who have applied for video viewing and is information on the number of viewers specified by the viewer (from among those presented by server20) when the viewer applies for video viewing at the user terminal30.

The item “viewer information” is information that identifies viewers other than the viewer who applied for video viewing when the item “number of viewers available” is multiple. For example, photo data that identifies the viewer's face is stored, similar to the item “face photo data” in the viewer database2021.

The item “state information” is information indicating the state of the viewer. Specifically, for example, it is information indicating the viewer's facial orientation when the viewer is in front of the display device, facial expressions, the viewer's body posture, and whether the viewer is in a wakeful or sleep state.

When the viewing acceptance module2033receives an application for video viewing from a viewer, the application information is stored in the viewing purchase database2023.

The following describes the video screening process by the video screen presentation system1in the Embodiment 1 with reference toFIG.7andFIG.8.

FIG.7is a flowchart showing an example of the flow of the video viewing application acceptance process by the screen presentation system1of the Embodiment 1.

In step S111, the input device33of the user terminal30receives an input operation of user information from the user to use the service of the screen presentation system1. For example, the user refers to a web page provided by the server20and operates to enter the user's identification information (viewer ID) and password. The communication IF32transmits the user's identification information and password information to the server20. The operation of the screen presentation system1may be require membership registration for video viewing. In this case, if the user is not registered as a member, the user's personal information is required to be entered on the web page for membership registration.

In step S121, the viewing acceptance module2033of the server20accepts the user's identification information and password information transmitted from the terminal device10via the communication unit201. The viewing acceptance module2033transmits to the accepted user's user terminal30, via the communication unit201, information on a list of images that can be viewed, information on the start date and time of viewing of each image and the price of the ticket, and an input format when the user wishes to view one of these images.

In step S112, the communication IF32of the user terminals30accepts the list information of viewable images transmitted from the server20, information such as the start date and time of viewing, and the input format. The output device34of the user terminals30displays the accepted input format on the screen. The input device33of the user terminals30accepts the input operation of the application for video viewing by the user from the displayed input format. The communication IF32transmits the application information for video viewing to the server20.

In step S122, the viewing acceptance module2033of the server20accepts the request information for video viewing transmitted from the terminal device10via the communication unit201.

In step S123, the server20sends the terminal device10to the user who made the application and whose application information for video viewing was received in step S122. The terminal device10does not necessarily be sent each time to a user who applied to view the video. If the user already possesses the terminal device10, this step can be omitted.

In step S124, the viewing acceptance module2033of the server20stores the user information received in step S121in the viewer database2021. In addition, the viewing acceptance module2033performs the process of storing the video viewing application information received in step S122in the viewing purchase database2023.

As described above, in the embodiment of screen presentation system1, when the user applies to view the video, the terminal device10is sent to the user. When the user receives the terminal device10, user connects it to own television set, as described below.

FIG.8is a flowchart showing an example of the flow of the video screening process by the screen presentation system1of the Embodiment 1.

In step S211, the user (viewer) who has made an application to view the video connects the connector14of the terminal device10sent by the operator of the server20to the user's own display device. The connection detection unit152of the terminal device10detects that the connector14of the terminal device10is connected to the display device. When the connector14is connected to the external connection port, the display device send a detection signal (or a response signal to the signal sent by the terminal device10to the display device) and the connection detection unit152detects that the connector14of the terminal device10is connected to the display device by the received signal.

In step S212, the photographing control unit153of the terminal device10activates the camera130when the connection of the terminal device to the display device is detected in step S211and causes the camera130to photograph the surroundings of the display device to generate the photographing image information. For example, when the photographing control unit153detects a person from the photographed image, it controls the photographing to continue with the photographed person as a viewer.

In step S213, the viewer identification information acquisition unit154of the terminal device10performs a process of transmitting to the server20and acquiring information identifying a viewer who has made an application for video viewing to the server20. The viewer identification information acquisition unit154causes information identifying the viewer to be transmitted to the server20via the transmission receiving unit151and acquires information identifying the viewer by a response from the server20.

In step S223, the viewer identification information transmission module2034of the server20transmits information identifying the viewer in response to the transmission request from the terminal device10in step S213. The viewer identification information transmission module2034transmits, as viewer identification information such as a code number or QR code, a facial photograph identifying a viewer's face, headcount information as information identifying the number of viewers and a facial photograph of all viewers of a plurality of viewers via communication unit201.

In step S214, the viewer identification unit156of the terminal device10identifies, from the image information of the viewer photograph by the camera130that started taking pictures in step S212, that the viewer taken is a legitimate viewer who has applied to request to the server20to view the video viewing. If the information identifying the viewer is a code number or QR code, the viewer identification unit156presents the code number or QR code so that the viewer can be photographed by the camera130, so that the information is read and judged to match. In addition, when the information identifying the viewer is a facial photograph identifying the viewer's face, the viewer identification unit156detects the viewer's face from the captured image information and determines whether it matches. When the information identifying the viewer is information identifying the number of viewers, the viewer identification unit156detects the viewer's face from the captured image information and determines whether it matches (or is less than or equal to) the permitted number of viewers.

In step S225, the video information distribution module2036of the server20distributes the video information pertaining to the video to the terminal device10of the viewer who made the application for video viewing. The video information distribution module2036transmits the video information to the terminal device10via the communication unit201.

In step S215, the video display control unit157of the terminal device10receives the video information transmitted from the server20in step S225and processes the various images to be shown (displayed) on the display device as video images. At this time, the video display control unit157causes the various images to be shown (displayed) on the display device only when it is authenticated in step S214that the viewer photographed by the camera130is a legitimate viewer who has made an application to the server20for video viewing.

In step S226, the server20stores information (illustration omitted) to the effect that the viewer who made an application for video viewing has viewed the video in the viewing purchase database2023.

As described above, in the embodiment of the screen presentation system1, when the viewer applies to view the video, the terminal device is sent to the viewer, and when the viewer connects the terminal device to the user's own display device, the viewer is authorized and the video is displayed on the display device. This prevents unauthorized use and makes it possible to construct a video distribution system that can be easily use.

4. Screen Example

The screen examples of the video screening process by the screen presentation system1are described below, with reference toFIG.9andFIG.10.

FIG.9shows an example screen displaying application information on the user terminal in the video viewing application acceptance process. The screen example inFIG.9shows a screen example in which information on one video is selected from the list of available video information and the input operation of the application for video viewing by the user is performed from the displayed input format. It corresponds to step S112ofFIG.7.

As shown inFIG.9, information on the video selected by the user is displayed on the output device (display)34of the user terminal30. In the example ofFIG.9, there is information on a video screening date and time1031aand selection buttons1031bto1031dthat allow the user to select the number of people who can view the video and the price of the ticket for that number of people. For example, “Solo” displayed on the selection button1031bindicates that one viewer can view the video, “Pair” displayed on the selection button1031cindicates that two viewers can view the video, and “Family” displayed on the selection button1031dindicates that four viewers can view the video. The user selects one of the selection buttons1031bto1031dand presses it by tapping, etc. to enable viewing of the selected video.

FIG.10shows an example of a state in which a viewer is filmed by the camera130of the terminal device10in the video screening process. The example inFIG.10shows an example of the filmed image information, in which a person is filmed by the camera130as a viewer under the control of the photographing control unit153. This is the captured image information according to step S212ofFIG.8.

As shown inFIG.10, a frame1032a, which is a shooting range by the camera130, is displayed in a window W, which shows an example of the shot image information by the camera130on a given PC or the like. The frame1032ais displayed with the viewer1032bcaptured and the viewer's face1032crecognized. Recognition of the face1032cis performed by a known technique of face recognition, as described above. In order to authenticate that the viewer1032bis a viewer who has made a legitimate application for video viewing, authentication is carried out, as described above, for example, by a feature extraction process.

As described above, according to this Embodiment, when a user request to view the video, a terminal device is sent to the user, and when the terminal device is connected to the user's display device, video information is distributed and the video is displayed on the display device. This makes it possible to easily use the video distribution system and view the video.

When the user connects the terminal device to the user's own display device, the viewer is photographed and the viewer is authenticated, and the video is displayed on the display device only when the viewer is authenticated to be a legitimate viewer. This makes it possible to prevent unauthorized use in the video distribution system.

Another Embodiment of the screen presentation system1is described below.

<1. Overall Configuration of the Screen Presentation System1>

FIG.11shows the functional configuration of the server20that constitutes the screen presentation system1in the Embodiment 2. The overall configuration of the screen presentation system1in the Embodiment 2 and the configuration of the terminal unit10are similar to those of the Embodiment 1 and will not be explained repeatedly. The configuration of the server20is similar to the Embodiment 1, except that it is newly equipped with a viewer state analysis module2037function, as shown inFIG.11. The functions of the viewer state analysis module2037in the Embodiment 2 are described below.

The viewer state analysis module2037controls the process of analyzing and obtaining information indicating the viewer's viewing state based on information indicating the viewer's state by the viewer state information acquisition unit155. The objects to be analyzed by the viewer state information acquisition unit155are, specifically, the direction of the viewer's face, the viewer's facial expression, and the viewer's body posture and motion, which are analyzed by image analysis. Information indicating the viewer's viewing state is, for example, the viewer's emotions, the viewer's degree of concentration on the image, and the viewer's degree of satisfaction with the image, such as whether the viewer is concentrating on viewing the image by looking in the direction of the display unit or not concentrating on viewing the image by looking in another direction or frequently moving his/her body (i.e. the viewer is not concentrating on viewing the image by looking in another direction or moving his/her body). The viewer is judged according to whether he or she is not concentrating on viewing the image (i.e. bored with viewing the image) by looking in other directions or moving his or her body frequently. From the information indicating the state of the viewer, whether the viewer is temporarily out of the picture (e.g. temporarily leaving the seat to go to the bathroom) or whether the viewer is out of the picture for a long period of time (i.e. has stopped viewing due to other business, etc.) are also be detected. The results of these analyses are recorded as the viewer's preference information for the video, and can be used to evaluate the video, recommend the video to the viewer and select the video to be provided by the operator.

The method of analyzing the orientation of the face by the viewer state analysis module2037may use the known technique of face recognition, escribed above. The method of analyzing facial expressions by the viewer state analysis module2037is, for example, a known method such as the Facial Action Coding System (FACS). This FACS is a method in which a code called an action unit is assigned to each facial expression muscle, and the emotion at that time is identified in response to the intensity and balance of the action unit that changes in response to the movement of the facial expression muscle and is an artificial intelligence technology for recognizing human emotions. This enables to identify the emotions of the viewer.

Information indicating the viewer's viewing state by the viewer state analysis module2037is acquired, for example, by quantifying the movement of the viewer's face changing direction, the viewer's facial expression, the viewer's body posture and the number and size of movements respectively.

As a example, the number of movements that change the direction of the viewer's face, the number of movements that change the viewer's body posture and the number of movements per predetermined time is calculated. Here, it is known empirically that when a person is concentrating on something, the number of face orientation and body movements tends to decrease. Therefore, it is possible to estimate the viewer's degree of concentration by analyzing the frequency of facial orientation and body movements. This enables to obtain information indicating the degree of concentration on the video image.

As another example, the degree of eye opening and the number of eye blinking at a given time are calculated. Here, it is known empirically that when a person is concentrating on something, the opening of the eyes tends to increase, and the number of eye blinking decreases. Therefore, it is considered possible to estimate the viewer's degree of concentration by analyzing the degree of eye opening and the frequency of the number of blinks per predetermined time. This enables to obtain information indicating the degree of concentration on the video image.

As another example, the degree of concentration may be calculated by performing predetermined operations on the movements of the viewer's face changing direction, the movements of the viewer's body changing posture, the degree of eye opening and the number of eye blinking per predetermined time, respectively, and adding the respective values after the operations. Furthermore, information indicating satisfaction with the images may be calculated from the information indicating the degree of concentration on the images. As the degree of satisfaction with the images is considered to be proportional to the degree of concentration on the images, the degree of concentration on the images may be regarded as the degree of satisfaction with the images, and the degree of satisfaction with the images may be calculated by performing a predetermined calculation on the degree of concentration on the images, taking the viewer's emotions into account.

The data structure in the Embodiment 2 is the same as that of the Embodiment 1.

The screening process of images by the screen presentation system1in the Embodiment 2 is described below, with reference toFIG.12.

FIG.12is a flowchart showing an example of the flow of the screening process of video images by the screen presentation system1of the Embodiment 2. The video screening process in the flowchart shown inFIG.12adds steps S311, S321, S322and S323as subsequent processes to steps S215and S225of the video screening process in the flowchart shown inFIG.8, so that overlapping processes will not be repeated and explained. In the flowchart shown inFIG.12, steps S211to S214and step S223are omitted.

In step S311, the viewer state information acquiring unit155of the terminal device10detects the viewer state from the image information of the viewer captured by the camera130and acquire the information indicating the viewer state. The viewer state information acquiring unit155also processes the information indicating the viewer state to be sent to the server20via the transmission receiving unit151.

In step S321, the viewer state analysis module2037of the server20receives the information indicating the viewer state transmitted from the terminal device10via the communication unit201.

In step S322, the viewer state analysis module2037of the server20carries out an analysis based on the information indicating the viewer state accepted in step S321and obtains information indicating the viewer's viewing state.

In step S323, the viewer state analysis module2037of the server20processes storing the information indicating the viewer state accepted in step S321to the viewing purchase database2023. The viewer state analysis module2037also processes storing information on the results of the analysis performed in step S322to the viewer database2021.

As described above, in the embodiment of the screen presentation system1, when a viewer views a video, the camera130of the terminal device10takes a picture of the viewer and obtains information indicating the viewer's state. Based on this information indicating the viewer's state, analysis is performed and information indicating the viewer's viewing state is acquired. This information is recorded as the viewer's preference information for images and can be used for recommending images to the viewer and selecting images to be provided by the operator concerned.

As described above, according to this Embodiment, when a viewer watches a video, the viewer is filmed and information indicating the viewer's state is acquired. Based on this information indicating the viewer's state, analysis is performed and information indicating the viewer's viewing state is obtained. This information is recorded as viewer preference information for the viewer's images and can be used for recommending images to the viewer and selecting images to be provided by the operator concerned.

Another Embodiment of the screen presentation system1is described below.

<1. Overall Configuration of the Screen Presentation System1>

FIG.13shows the functional configuration of the server20that constitutes the screen presentation system1of the Embodiment 3. The overall configuration of the screen presentation system1in the Embodiment 3 and the configuration of the terminal unit10are similar to those of the Embodiment 1 and will not be explained repeatedly. The configuration of the server20is similar to that of the Embodiment 1, except that it is newly equipped with a function of the evaluation information acceptance module2038, as shown inFIG.13. The function of the evaluation information acceptance module2038in the Embodiment 3 is described below.

The evaluation information acceptance module2038controls the process of accepting the evaluation information sent by the viewer on the user terminals30, when the viewer sends evaluation information on the viewed video. The evaluation information transmitted by the viewer is, for example, information indicating the viewer's impression of the viewed video as interesting, enjoyable, boring or tedious, and may be information in the form of free descriptions in text format, or a numerical value of the evaluation, for example, one of 1 to 5 in a 5-level evaluation. This evaluation information is sent to the producers, performers, etc. who produced the video. Furthermore, the evaluation information may be an item with monetary value that can be sent to the producers, performers, etc. who produced the video in accordance with the evaluation of the video. Here, the item with monetary value may be a cash settlement information, credit information, various point information and cryptographic assets that can be settled in the same way as cash.

As described above, according to this Embodiment, when a viewer watches a video, the viewer sends evaluation information for the video, which is accepted. This evaluation information is transmitted to the producers, performers, etc. who produced the vide. This enables the feedback of the viewer's impressions to the producers, etc. If the evaluation information is something that has monetary value, it is possible to provide an item with a monetary value to the producers of the good video in the form of so-called throwaway money or a small tip.

The Embodiments pertaining to the disclosure have been described above. They can be implemented in various other forms, and can be implemented with various omissions, substitutions, and changes. These Embodiments and variations, as well as omissions, substitutions, and modifications, are included within the technical scope of the claims and their equivalents.

APPENDIX

The matters described in each of the above-mentioned forms are appended below.

A terminal device connected to a display device displaying video comprising a control unit and a photographing unit; wherein the photographing unit captures an image of a viewer viewing the video displayed on the display device and generates captured image information; and wherein the control unit executes a step of detecting the connection of the terminal device to the display device; a step of acquiring information indicating a state of the viewer based on the captured image information; and a step of receiving video information distributed from an external source and displaying the video on the display device based on the state of the viewer.

The terminal device according to appendix 1, wherein the control unit is configured to execute a step of detecting a state in which the viewer is ready to view the video based on the captured image information and acquiring information indicating the viewer's state, and a step of starting the display of the video on the display device when the state in which the viewer is ready to view the video.

The terminal device according to appendix 2, wherein the terminal device is configured to detect that the viewer is ready to view the video by detecting at least one of the following in the captured image information: the viewer's facial direction and facial expression, the viewer's posture, and the viewer's awake state.

The terminal device according to any one of appendixes 1 to 3, wherein the control unit is further configured to execute a step of acquiring information from an external source identifying the viewer allowed to view the video, and a step of starting the display of the video on the display device when the viewer is identified as a person allowed to view the video by a predetermined authentication based on the information identifying the viewer.

The terminal device according to appendix 4, wherein the information identifying the viewer as a person allowed to view the video is transmitted by the viewer from an external device, and when the terminal device receives the information transmitted by the viewer, the predetermined authentication is performed and the terminal device executes the step of starting the display of the video on the display device.

The terminal device according to appendix 4 or appendix 5, wherein the control unit is configured to execute a step of detecting the viewer based on the captured image information, and a step of starting the display of the video on the display device when the viewer detected on the basis of the captured image information is authenticated as a person allowed to view the video, as a predetermined authentication.

The terminal device according to appendix 6, wherein the control unit is configured to execute a step of acquiring viewer image information identifying the face of the viewer allowed viewing the video, a step of detecting the face of the viewer based on the captured image information, and a step of starting the display of the video on the display device when the detected viewer is authenticated as a person allowed to view the video based on a comparison of the viewer's image information and the viewer's face in the captured image information.

The terminal device according to appendix 7, wherein the control unit is configured to execute, in the step of detecting the viewer's face, extracting a first feature quantity of the viewer's face in the viewer image information, and extracting a second feature quantity of the viewer's face in the viewer image information and comparing the first feature quantity and the second feature quantity, and if they match a predetermined value or a predetermined percentage or more, authenticating that the detected viewer is a person allowed to view the video, and starting to display the video on the display device.

The terminal device according to any one of appendixes 6 to 8, wherein the control unit is configured to execute a step of acquiring information identifying the number of viewers allowed to view the video, a step of detecting the number of viewers based on the captured image information, and a step of starting the display of the video on the display device when the detected number of viewers is authenticated to be less than or equal to the number of persons allowed to view the video.

The terminal device according to any one of appendixes 1 to 9, wherein the control unit is configured to execute a step of detecting the viewer when viewing the video based on the captured image information and a step of acquiring information indicating the viewer's viewing state based on the viewer's state in the captured image information.

The terminal device according to appendix 10, wherein the terminal device is configured to execute a step of detecting the face of the viewer when viewing the video based on the captured image information, a step of analyzing the facial expression of the viewer in the captured image information, and a step of acquiring information indicating the viewing state of the viewer.

The terminal device according to appendix 10 or appendix 11, wherein the terminal device is configured to execute a step of detecting the body motion of the viewer when viewing the video based on the captured image information, a step of analyzing the viewer's body motion in the captured image information, and a step of acquiring information indicating the viewer's viewing state.

The terminal device according to any one of appendixes 10 to 12, wherein the terminal device is configured to execute a step of acquiring, as information indicating the viewing state of the viewer, information indicating at least one of the following states: the viewer's emotion, the viewer's degree of concentration on the video, or the viewer's degree of satisfaction with the video.

The terminal device according to any one of appendixes 1 to 13, wherein the control unit is configured to execute a step of receiving video information distributed from external sources, a step of displaying the image of the video on the display device, and a step of outputting the sound of to the video.

The terminal device according to any one of appendixes 1 to 14, wherein the control unit is configured to execute a step of accepting evaluation information on the video from the viewer.

A method for execution by a computer provided with a processor and connected to a display device displaying video, wherein the method comprises a processor executing a step of detecting the connection of the terminal device to the display device; a step of capturing a viewer viewing the video displayed on the display device and generating the captured image information; a step of acquiring information indicating a state of the viewer based on the captured image information; and a step of receiving video information distributed from an external source and displaying the video on the display device on the state of the viewer.

A program for execution by a computer provided with a processor and connected to a display device displaying video, wherein the program is configured to cause the processor to execute a step of detecting the connection of the terminal device to the display device; a step of capturing a viewer viewing the video displayed on the display device and generating the captured image information; a step of acquiring information indicating a state of the viewer based on the captured image information; and a step of receiving image information related to the image delivered from an external source based on the information indicating the state of the viewer, and displaying the image on the said display device.10terminal device20server30user terminals80network130camera140storage unit141equipment information142viewer information143video information150control unit22communication IF23input/output IF25memory26storage29processor201communication unit202storage unit2021viewer database2022equipment database2023viewing purchase database2024video database203control unit