Patent Description:
There is a conventional technique of displaying an image captured by a camera provided at the entrance door of a building, on a display terminal in the building. For example, Patent Literature (PTL) <NUM> discloses a video door phone device for stable wireless communication between an indoor master unit and an entrance slave unit. With such a video door phone device, when, for example, a delivery person arrives at the door of a building and activates an intercom, an image of the delivery person captured by the entrance slave unit is displayed on the display of the indoor master unit. In addition, PTL <NUM> discloses an intercom device and PTL <NUM> discloses an integrated accessory control user interface.

With the above conventional technique, however, the indoor master unit only displays the image of the delivery person, and thus, it is not possible to effectively present the user with information which assists the user in determining the next action.

In view of this, the present disclosure provides a display control method and the related technologies capable of effectively presenting a user with information which assists the user in determining the next action when a delivery person arrives with a delivery item.

A display control method according to an aspect of the present disclosure is a display control method performed using a system including an information processing server, and the display control method includes: (a) obtaining, by the information processing server, using a sensor, a user state indicating whether a user is currently cooking using a cooker, when an image of a delivery person delivering an item to a building in which the user is present is captured by a camera provided at an entrance door of the building; (b) obtaining, by the information processing server, content information of the item; (c) obtaining, by the information processing server, at least one suggestion associated with the user state and the content information of the item; and (d) causing, by the information processing server, a display terminal provided in the building to display the image of the delivery person, the content information of the item, and the at least one suggestion.

Note that these general or specific aspects may be implemented using a system, a device, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or any combination of systems, devices, integrated circuits, computer programs, or recording media.

A display control method according to an aspect of the present disclosure is capable of effectively presenting a user with information which assists the user in determining the next action when a delivery person arrives with a delivery item.

<FIG> illustrates service type <NUM> (SaaS-based) according to Embodiment <NUM>.

Hereinafter, embodiments will be described in detail with reference to the accompanied drawings.

Note that the following embodiments each illustrate a general or specific example. The numerical values, shapes, materials, constituent elements, the arrangement and connection of the constituent elements, steps, the processing order of the steps, etc. illustrated in the following exemplary embodiments are mere examples, and are therefore not intended to limit the scope of the claims. Among the constituent elements in the following embodiments, those not recited in any one of the independent claims representing the most generic concepts will be described as optional constituent elements. Note that the drawings are represented schematically and are not necessarily precise illustrations. Throughout the drawings, constituent elements that are essentially the same share like reference signs, and overlapping descriptions thereof are omitted or simplified.

The following describes the knowledge forming the basis of an information processing system and method according to the present disclosure. The inventor has considered the actions that a user would take when a delivery person comes. When a delivery person comes, the user takes different actions depending on the user's situation. Meanwhile, the user is required to take a prompt action in response to the delivery person. In view of this, an information processing system and method according to the present disclosure provide the user with information about the delivery to make it easier for the user to determine the action that the user wants to take. Additionally, an information processing system and method according to the present disclosure simplify the user's action by providing the user with a suggestion to expedite the user's action, and controlling devices according to the user's response to the suggestion displayed. That is to say, the inventor has conceived an information processing system and method which display, on one screen, both information about the delivery and a suggestion for expediting the user's action, to facilitate both the user's understanding of the situation and the user's action.

First, configurations of an information processing system and peripheral devices according to Embodiment <NUM> will be described with reference to <FIG>. <FIG> is an external view of an information processing system and peripheral devices according to Embodiment <NUM>. <FIG> is a block diagram illustrating a functional configuration of the information processing system according to Embodiment <NUM>.

Information processing system <NUM> includes information processing server <NUM> and one or more display terminals including display terminal <NUM>. Information processing server <NUM> and one or more display terminals are communicably connected to one another. Hereinafter, display terminal <NUM> will be described as a representative of the one or more display terminals, and description of the rest of the display terminals will be omitted.

Information processing system <NUM> is communicably connected to camera <NUM>, electronic lock <NUM>, information terminal <NUM>, cooker <NUM>, sensor <NUM>, first database <NUM>, second database <NUM>, and third database <NUM>.

Information processing server <NUM> is, for example, a cloud server, and causes display terminal <NUM> to display at least one suggestion based on a user state.

Display terminal <NUM> includes a display, and displays, on the display, at least one suggestion received from information processing server <NUM>. In the present embodiment, display terminal <NUM> is provided on a wall surface in a building. Alternatively, display terminal <NUM> may be a mobile terminal.

Camera <NUM> is provided at entrance door <NUM> of the building. In <FIG>, camera <NUM> is built into a door phone. For example, camera <NUM> captures an image of delivery person <NUM> who delivers item <NUM> to the building in which user <NUM> is present. The building includes, for example, a residence, an apartment building, an office building having a plurality of offices, a commercial building having a plurality of shops, and a hotel having a plurality of rooms. Entrance door <NUM> of an apartment building may be an entrance door of each residence, or may be an entrance door of the apartment building. Entrance door <NUM> of an office building may be an entrance door of each office, or may be an entrance door of the office building. Likewise, entrance door <NUM> of a commercial building may be an entrance door of each shop, or may be an entrance door of the commercial building.

Electronic lock <NUM> is a lock capable of remote control on the opening and closing of entrance door <NUM> of the building. Electronic lock <NUM> is also referred to as an electronic key. Electronic lock <NUM> controls the opening and closing of entrance door <NUM> based on a lock/unlock signal from information processing system <NUM>.

Information terminal <NUM> is a terminal usable by delivery person <NUM>. Information terminal <NUM> is a handy terminal carried by delivery person <NUM>, for example. Information terminal <NUM>, for example, receives a request for redelivery of item <NUM> from information processing system <NUM>. Information terminal <NUM> may be provided at entrance door <NUM>, for example.

Cooker <NUM> is a device used when user <NUM> cooks. Cooker <NUM> is, for example, a microwave oven, a cooking stove, or the like. Cooker <NUM> can control power ON and OFF based on a signal from information processing system <NUM>.

Sensor <NUM> transmits to information processing system <NUM> a sensor value used for obtaining a user state indicating whether user <NUM> is currently cooking using cooker <NUM>. Specifically, sensor <NUM> is, for example, an image sensor which is provided in the building and captures an image of the inside of the building. Sensor <NUM> may be, for example, an ON/OFF sensor which detects power ON and OFF of cooker <NUM>.

First database <NUM> is a storage device storing first data 801a which associates sensor values and user states for each of a plurality of users. Specifically, first database <NUM> is, for example, a database server which includes a processor, memory, and a hard disk drive.

Here, first data 801a will be described using a specific example. <FIG> illustrates an example of first data 801a according to Embodiment <NUM>.

In <FIG>, first data 801a includes user information, sensor value, and user state. User information is information that identifies users. Sensor value represents an image pattern, for example. User state indicates whether a user is currently cooking. According to first data 801a in <FIG>, the user state is determined to be cooking when Pattern <NUM> is included in an image of the user identified as USER <NUM>.

Second database <NUM> is a storage device storing second data 802a which associates (i) a plurality of pieces of identification information or a plurality of scheduled delivery dates and times and (ii) pieces of content information of a plurality of items. Specifically, second database <NUM> is, for example, a database server which includes a processor, memory, and a hard disk drive.

Here, second data 802a will be described using a specific example. <FIG> illustrates an example of second data 802a according to Embodiment <NUM>.

In <FIG>, second data 802a includes ID, scheduled delivery date and time, image information, and urgent flag. ID is an example of identification information for identifying content information. Scheduled delivery date and time indicates a date and time of scheduled delivery of an item.

Image information is identifier information which identifies an image of an item (for example, a uniform resource identifier (URI)). Urgent flag is an example of urgent information indicating whether the delivery of the item is urgent. For example, the urgent flag is set by the user when the user purchases an item via a website. Here, urgent flag <NUM> indicates that the delivery is urgent, whereas urgent flag <NUM> indicates that the delivery is non-urgent. The image information and the urgent flag correspond to the content information of the item.

It can be seen from second data 802a in <FIG> that, for example, the image of an item identified by <NUM> is. jpg, and that the delivery of this item is urgent. It can also be seen that the image of an item scheduled to be delivered on <NUM>/<NUM>/<NUM> at <NUM>:<NUM> is. jpg, and that the delivery of this item is urgent.

Third database <NUM> is a storage device storing third data 803a which associates (i) a plurality of combinations of a user state and content information of an item and (ii) a plurality of suggestion sets each including at least one suggestion. Specifically, third database <NUM> is, for example, a database server which includes a processor, memory, and a hard disk drive.

Here, third data 803a will be described using a specific example. <FIG> illustrates an example of third data 803a according to Embodiment <NUM>.

Third data 803a in <FIG> includes user state, urgent flag, first suggestion, and second suggestion. User state indicates whether the user is currently cooking. As in <FIG>, urgent flag is an example of urgent information indicating whether the delivery of the item is urgent. According to third data 803a in <FIG>, when, for example, the user state is Cooking and the urgent information indicates Yes, the first suggestion of Cooker OFF and the second suggestion of Unlock are obtained.

Next, a configuration of information processing server <NUM> will be described. As illustrated in <FIG>, information processing server <NUM> includes processor <NUM>, memory <NUM>, and communication unit <NUM>.

Processor <NUM> is an electric circuit connected to memory <NUM>. When an image of delivery person <NUM> delivering item <NUM> to the building in which user <NUM> is present is captured by camera <NUM> provided at entrance door <NUM> of the building, processor <NUM> obtains, using sensor <NUM>, a user state indicating whether user <NUM> is currently cooking using cooker <NUM>. Processor <NUM> further obtains content information of item <NUM>. Processor <NUM> then obtains at least one suggestion associated with the user state, and causes display terminal <NUM> to display the image of delivery person <NUM>, the content information of item <NUM>, and at least one suggestion. The details of the processing will be described later using a flow chart etc..

Memory <NUM> is, for example, a semiconductor memory and/or a hard disk drive. Memory <NUM> can store a software program or an instruction. When the software program or the instruction stored in memory <NUM> is executed, processor <NUM> can perform the processing described above.

Communication unit <NUM> is a network adapter for performing one or both of wired communication and wireless communication with display terminal <NUM>, camera <NUM>, electronic lock <NUM>, information terminal <NUM>, cooker <NUM>, sensor <NUM>, first database <NUM>, second database <NUM>, and third database <NUM> via a communication network. The communication network may be, for example, the Internet, a wide area network (WAN), a local area network (LAN), or any combination of these. Wireless communication may use wireless LAN, Bluetooth (registered trademark), ZigBee (registered trademark), or any combination of these.

Next, a configuration of display terminal <NUM> will be described. As illustrated in <FIG>, display terminal <NUM> includes processor <NUM>, memory <NUM>, communication unit <NUM>, display <NUM>, and input unit <NUM>.

Processor <NUM> is an electric circuit connected to memory <NUM>. Processor <NUM> displays, on display <NUM>, an image of delivery person <NUM>, content information of item <NUM>, and at least one suggestion which are received from information processing server <NUM>.

Memory <NUM> is, for example, a semiconductor memory and/or a hard disk drive. Memory <NUM> can store a software program or an instruction. When the software program or the instruction stored in memory <NUM> is executed, processor <NUM> performs various processing.

Communication unit <NUM> is a network adapter for performing one or both of wired communication and wireless communication with information processing server <NUM> via a communication network.

Display <NUM> displays the image of delivery person <NUM>, the content information of item <NUM>, and at least one suggestion which are received from information processing server <NUM>. For example, display <NUM> is a liquid crystal display or an organic EL display.

Input unit <NUM> is an input device used for receiving user input. For example, input unit <NUM> is a touch panel (that is, a touch screen) provided integrally with display <NUM>. Input unit <NUM> may be, for example, a mechanical button, a mouse, a keyboard, or any combination of these and a touch panel.

Next, processing of information processing system <NUM> configured as described above will be described with reference to <FIG>. <FIG> is a flow chart illustrating processing of the information processing system according to Embodiment <NUM>.

Processor <NUM> of information processing server <NUM> obtains an image of delivery person <NUM> from camera <NUM> (hereinafter also referred to as a camera image). <FIG> is an example of camera image <NUM> according to Embodiment <NUM>. In <FIG>, camera image <NUM> includes delivery person <NUM> and item <NUM>. Camera image <NUM> also includes sheet <NUM> on which identification information identifying content information of item <NUM> is provided. For example, the identification information is provided on sheet <NUM> in the form of a bar code and/or a character string.

Processor <NUM> of information processing server <NUM> obtains the content information of item <NUM>. Specifically, processor <NUM> first obtains the identification information from camera image <NUM>. Then, processor <NUM> obtains the content information of item <NUM> based on the obtained identification information, by referring to second data 802a. For example, processor <NUM> obtains <NUM> as the identification information by reading the bar code provided on sheet <NUM> of camera image <NUM>. Then, by referring to second data 802a in <FIG>, processor <NUM> obtains, as the content information of item <NUM>, image information. jpg and urgent flag <NUM> which are associated with <NUM>.

Note that processor <NUM> does not necessarily need to obtain the content information of item <NUM> based on the identification information. For example, processor <NUM> may obtain the content information of item <NUM> based on the date and time of obtainment of the image, by referring to second data 802a. For example, when camera image <NUM> was captured at <NUM>:<NUM> on October <NUM>, <NUM>, processor <NUM> may obtain, as the content information of item <NUM>, image information. jpg and urgent flag <NUM> which are associated with the scheduled delivery date and time <NUM>/<NUM>/<NUM><NUM>:<NUM> which is closest to the actual capturing date and time.

Processor <NUM> of information processing server <NUM> obtains a sensor value that is an output value of sensor <NUM>. When sensor <NUM> is an image sensor, for example, processor <NUM> obtains image data as the sensor value. When sensor <NUM> is an ON/OFF sensor, for example, processor <NUM> obtains an ON/OFF signal as the sensor value.

Processor <NUM> of information processing server <NUM> obtains a user state using sensor <NUM>. Specifically, processor <NUM> obtains a user state based on a sensor value obtained by sensor <NUM>, for example. More specifically, processor <NUM> obtains a user state based on a sensor value by referring to first data 801a. At this time, processor <NUM> may identify the user based on the sensor value, and obtain a user state based on the identified user and the sensor value by referring to first data 801a.

For example, when sensor <NUM> is an image sensor, processor <NUM> recognizes the user based on image data. The method for user recognition is not particularly limited, and a conventional technique may be used. For example, processor <NUM> identifies user <NUM> as USER <NUM>. Here, when the image data includes Pattern <NUM>, processor <NUM> obtains, by referring to first data 801a in <FIG>, a user state indicating that user <NUM> is currently cooking using cooker <NUM>.

When, for example, sensor <NUM> is an ON/OFF sensor and processor <NUM> obtains an ON signal, processor <NUM> obtains a user state indicating that user <NUM> is currently cooking using cooker <NUM>. On the other hand, when processor <NUM> obtains an OFF signal, processor <NUM> obtains a user state indicating that user <NUM> is not cooking using cooker <NUM>.

Processor <NUM> of information processing server <NUM> determines at least one suggestion. Specifically, processor <NUM> obtains at least one suggestion associated with the user state and the content information of item <NUM>. For example, processor <NUM> obtains, by referring to third data 803a, at least one suggestion associated with the obtained user state and content information of item <NUM>.

For example, when the user state indicates that the user is currently cooking, and the urgent information included in the content information of item <NUM> indicates that the delivery is urgent, processor <NUM> obtains, by referring to third data 803a in <FIG>, a first suggestion to turn off cooker <NUM> and a second suggestion to unlock electronic lock <NUM> on entrance door <NUM> of the building. For example, when the user state indicates that the user is currently cooking, and the urgent information indicates that the delivery is non-urgent, processor <NUM> obtains, by referring to third data 803a in <FIG>, a first suggestion to request redelivery of item <NUM> and a second suggestion to receive item <NUM>. For example, when the user state indicates that the user is not cooking, and the urgent information indicates that the delivery is urgent, processor <NUM> obtains, by referring to third data 803a in <FIG>, a first suggestion to unlock electronic lock <NUM> on entrance door <NUM> of the building. In other words, in this case, processor <NUM> obtains a single suggestion. For example, when the user state indicates that the user is not cooking, and the urgent information indicates that the delivery is urgent, processor <NUM> obtains, by referring to third data 803a in <FIG>, a first suggestion to receive item <NUM> and a second suggestion to request redelivery of item <NUM>.

Processor <NUM> of information processing server <NUM> causes display terminal <NUM> to display at least one suggestion determined. For example, processor <NUM> transmits screen data including at least one suggestion to display terminal <NUM>. Here, specific examples of a screen displayed by display terminal <NUM> will be described with reference to <FIG>.

<FIG> illustrates an example of a first screen according to Embodiment <NUM>. The first screen is displayed when the user state indicates that the user is currently cooking and the urgent information indicates that the delivery is urgent.

As illustrated in <FIG>, display <NUM> displays, in sequence from the left to the right, camera image <NUM>, image <NUM> of item <NUM>, first suggestion <NUM> to turn off cooker <NUM>, and second suggestion <NUM> to unlock electronic lock <NUM> on entrance door <NUM>. Here, when an operation to select first suggestion <NUM> and/or second suggestion <NUM> is performed, device control corresponding to the selected suggestion is performed. That is to say, the screen of display <NUM> is segmented. The screen includes a region for displaying a camera image, a region for displaying the content of an item, and a region for displaying at least one suggestion. Display <NUM> also displays, at the top, a current date and time, a login user, and various states. The display of such information is a mere example, and is not limited to this example.

Camera image <NUM>, image <NUM> of item <NUM>, and first and second suggestions <NUM> and <NUM> are respectively displayed in three display regions arranged horizontally on display <NUM>. Here, the three display regions are identical in shape and size. Each of the three display regions is a square region occupying about one third of the display area of display <NUM>. Note that being identical encompasses being substantially identical as well as being strictly identical. In other words, being identical means being approximately identical. Furthermore, a square encompasses a precise square and shapes which can be considered substantially identical to a precise square. For example, the screen may include three display regions only. In a more limited example, the screen of display <NUM> may only include three display regions that are identical in size and shape.

First suggestion <NUM> and second suggestion <NUM> are arranged next to each other vertically. Here, first suggestion <NUM> is displayed with higher priority by being displayed above second suggestion <NUM>. Note that the method for display with priority is not limited to the vertical arrangement. For example, first suggestion <NUM> may be displayed with higher priority by being displayed in a different size or color.

Camera image <NUM>, image <NUM> of item <NUM>, and first and second suggestions <NUM> and <NUM> may be displayed on display <NUM> simultaneously, for example. In other words, the three pieces of information may simultaneously start being displayed. Alternatively, for example, after camera image <NUM> is displayed, image <NUM> of item <NUM> and first and second suggestions <NUM> and <NUM> may be displayed with camera image <NUM>. This way, image <NUM> of item <NUM> and first and second suggestions <NUM> and <NUM> are displayed after the user recognizes camera image <NUM>, thus assisting the user to grasp the current situation in sequence. For example, when the user sees camera image <NUM> and recognizes that a delivery item has arrived, the user considers what to do next. By displaying image <NUM> of item <NUM> and first and second suggestions <NUM> and <NUM> at this timing, it is possible to provide the user with information appropriate to the user's situation. For example, since the user in, for example, a residence or an apartment building uses the system on a daily basis and becomes familiar with the information displayed on display <NUM>, camera image <NUM>, image <NUM> of item <NUM>, and first and second suggestions <NUM> and <NUM> may be displayed simultaneously if the user has used the system for a certain period or longer. Examples of the certain period include one week, one month, and three months. In the case of an office building, a commercial building, or a hotel, for example, the user may not use the system on a daily basis, and thus may be unfamiliar with the information displayed on display <NUM>. In the case of an office building, a commercial building, or a hotel, for example, camera image <NUM> may be displayed first, and image <NUM> of item <NUM> and first and second suggestions <NUM> and <NUM> may be displayed next, so that even the unfamiliar user can easily understand the information. Note that although the example above is the case of displaying image <NUM> of item <NUM> and first and second suggestions <NUM> and <NUM> simultaneously, camera image <NUM>, image <NUM> of item <NUM>, and first and second suggestions <NUM> and <NUM> may be separately displayed sequentially.

The following describes the meaning of camera image <NUM>, image <NUM> of item <NUM>, and first and second suggestions <NUM> and <NUM> included in the first screen. Camera image <NUM> is included to allow user <NUM> to recognize arrival of delivery person <NUM>, when delivery person <NUM> arrives at entrance door <NUM> of the building. Image <NUM> of item <NUM> is included to show the content of item <NUM> delivered by delivery person <NUM>. This allows user <NUM> to recognize item <NUM> that is being delivered by delivery person <NUM>.

The first screen includes first suggestion <NUM> and second suggestion <NUM> to provide guidance on the next action of user <NUM> in response to camera image <NUM> and image <NUM> of item <NUM>. Because user <NUM> is highly likely to receive item <NUM>, first suggestion <NUM> and second suggestion <NUM> are displayed as the next action of user <NUM> determined based on image <NUM> of item <NUM>. That is to say, first suggestion <NUM> and second suggestion <NUM> are included based on the presumption that user <NUM> is highly likely to turn off cooker <NUM> and/or unlock electronic lock <NUM> to receive urgent item <NUM>.

In other words, the first screen includes camera image <NUM> and image <NUM> of item <NUM> to allow user <NUM> to easily check item <NUM> delivered to user <NUM>. Further, the first screen includes first suggestion <NUM> and second suggestion <NUM> as the next action of user <NUM> determined based on camera image <NUM> and image <NUM> of item <NUM>. By seeing the first screen, user <NUM> can check, all at once, the arrival of delivery person <NUM> delivering item <NUM>, the content of item <NUM>, and the next action, and can then take an action.

<FIG> illustrates an example of a second screen according to Embodiment <NUM>. The second screen is displayed when the user state indicates that the user is currently cooking and the urgent information indicates that the delivery is non-urgent.

As illustrated in <FIG>, display <NUM> displays, in sequence from the left to the right, camera image <NUM>, image <NUM> of item <NUM>, first suggestion <NUM> to request redelivery of item <NUM>, and second suggestion <NUM> to receive item <NUM>. Here, when an operation to select first suggestion <NUM> is performed, a third screen, which will be described later, is displayed on display <NUM>, for example. On the other hand, when an operation to select second suggestion <NUM> is performed, the first screen in <FIG> is displayed, for example.

By seeing the second screen, user <NUM> can check, all at once, the arrival of delivery person <NUM> delivering item <NUM>, the content of item <NUM>, and the next action, and can then take an action.

<FIG> illustrates an example of the third screen according to Embodiment <NUM>. The third screen is displayed after the user has selected the suggestion to request redelivery of item <NUM>.

As illustrated in <FIG>, display <NUM> displays, in sequence from the left to the right, camera image <NUM>, image <NUM> of item <NUM>, and graphical user interface (GUI) <NUM> via which user <NUM> can select a redelivery time and transmit a redelivery request to information terminal <NUM> of delivery person <NUM>. User <NUM> can select a redelivery time by operating GUI <NUM>. For example, by pressing GUI <NUM>, it is possible to transmit a redelivery request to information terminal <NUM>, requesting delivery person <NUM> to redeliver item <NUM> after the selected number of hours. That is to say, a request for redelivery of item <NUM> is transmitted to information terminal <NUM> when a selection of the suggestion to request redelivery of item <NUM> is received.

<FIG> illustrates an example of a fourth screen according to Embodiment <NUM>. The fourth screen is displayed when the user state indicates that the user is not cooking and the urgent information indicates that the delivery is urgent.

As illustrated in <FIG>, display <NUM> displays, in sequence from the left to the right, camera image <NUM>, image <NUM> of item <NUM>, and first suggestion <NUM> to unlock electronic lock <NUM> on entrance door <NUM>. Here, when an operation to select first suggestion <NUM> is performed, electronic lock <NUM> is unlocked.

By seeing the fourth screen, user <NUM> can check, all at once, the arrival of delivery person <NUM> delivering item <NUM>, the content of item <NUM>, and the next action, and can then take an action.

<FIG> illustrates an example of a fifth screen according to Embodiment <NUM>. The fifth screen is displayed when the user state indicates that the user is not cooking and the urgent information indicates that the delivery is non-urgent.

As illustrated in <FIG>, display <NUM> displays, in sequence from the left to the right, camera image <NUM>, image <NUM> of item <NUM>, first suggestion <NUM> to receive item <NUM>, and second suggestion <NUM> to request redelivery of item <NUM>. Here, when an operation to select first suggestion <NUM> is performed, the fourth screen in <FIG> is displayed, for example. On the other hand, when an operation to select second suggestion <NUM> is performed, the third screen in <FIG> is displayed, for example.

By seeing the fifth screen, user <NUM> can check, all at once, the arrival of delivery person <NUM> delivering item <NUM>, the content of item <NUM>, and the next action, and can then take an action.

Processor <NUM> of information processing server <NUM> performs control based on the suggestion selected. For example, processor <NUM> transmits an unlock signal to electronic lock <NUM> on entrance door <NUM> when display terminal <NUM> receives a selection of the suggestion to unlock electronic lock <NUM> (second suggestion <NUM> in <FIG>, for example). This way, electronic lock <NUM> is unlocked. For example, an OFF signal is transmitted to cooker <NUM> when display terminal <NUM> receives a selection of the suggestion to turn off cooker <NUM> (first suggestion <NUM> in <FIG>, for example). This way, cooker <NUM> becomes non-operational. When entrance door <NUM> is an entrance door in a public space of, for example, an apartment building, an office building, or a commercial building, electronic lock <NUM> may be unlocked after a lapse of a first period or less, following the reception of the unlock signal. An example of the first period is a period required for communication, or a few seconds. However, when entrance door <NUM> is an entrance door leading directly to a private space such as a residence, an office, or a shop, delivery person <NUM> can enter the private space before the user arrives at entrance door <NUM>, and thus, the user may not feel safe using the system. In view of this, as compared to electronic lock <NUM> on entrance door <NUM> to a public space, electronic lock <NUM> on entrance door <NUM> to a private space may be unlocked after a lapse of a second period which is longer than the first period, following the reception of the unlock signal. An example of the second period is an average period required for the user to come to entrance door <NUM>, or <NUM> seconds. Alternatively, the second period may be user-settable. For example, in the case of a residence, the second period may be set to a time period corresponding to the distance from the user to the kitchen.

As described above, in information processing system <NUM> according to the present embodiment, information processing server <NUM> can: (a) obtain, using sensor <NUM>, a user state indicating whether user <NUM> is currently cooking using cooker <NUM>, when an image of delivery person <NUM> delivering item <NUM> to a building in which user <NUM> is present is captured by camera <NUM> provided at entrance door <NUM> of the building; (b) obtain content information of item <NUM>; (c) obtain at least one suggestion associated with the user state and the content information of item <NUM>; and (d) cause display terminal <NUM> provided in the building to display the image of delivery person <NUM>, the content information of item <NUM>, and the at least one suggestion.

This makes it possible to display, on display terminal <NUM>, at least one suggestion according to the user state and the content information of item <NUM>. That is to say, at least one suggestion appropriate to user <NUM> and item <NUM> can be presented to user <NUM>. Moreover, since the content information of item <NUM> can be displayed in addition to the image of delivery person <NUM>, user <NUM> can determine the next action based on the content of item <NUM>. Accordingly, when delivery person <NUM> delivering item <NUM> arrives, it is possible to effectively present user <NUM> with information which assists user <NUM> in determining the next action.

In information processing system <NUM> according to the present embodiment, the content information of item <NUM> includes urgent information indicating whether delivery of item <NUM> is urgent, and in (c), when the user state indicates that user <NUM> is currently cooking and the urgent information indicates that the delivery is urgent, information processing server <NUM> can obtain, as the at least one suggestion, a suggestion to turn off cooker <NUM> and a suggestion to unlock electronic lock <NUM> on entrance door <NUM> of the building.

With this, when item <NUM> of an urgent delivery arrives while user <NUM> is cooking, it is possible to effectively present the user with suggestions of actions necessary for safely receiving item <NUM>. That is to say, it is possible to present user <NUM> with a suggestion of an action that reduces occurrence of an accident with cooker <NUM> while inhibiting a postponement of reception of item <NUM> of the urgent delivery, and it is possible to more effectively present user <NUM> with information which assists user <NUM> in determining the next action.

In information processing system <NUM> according to the present embodiment, in (c), when the user state indicates that user <NUM> is currently cooking and the urgent information indicates that the delivery is non-urgent, information processing server <NUM> can obtain, as the at least one suggestion, a suggestion to request redelivery of item <NUM> and a suggestion to receive item <NUM>.

With this, when item <NUM> of a non-urgent delivery arrives while user <NUM> is cooking, it is possible to present a suggestion to request redelivery of item <NUM> and a suggestion to receive item <NUM>, from which user <NUM> can select the next action. That is to say, with reference to the content information of item <NUM>, user <NUM> can determine, for each item <NUM>, whether to request redelivery of item <NUM> or to receive item <NUM>. In other words, information processing server <NUM> can more effectively present user <NUM> with information which assists user <NUM> in determining the next action.

In information processing system <NUM> according to the present embodiment, in (c), when the user state indicates that user <NUM> is not cooking and the urgent information indicates that the delivery is urgent, information processing server <NUM> can obtain, as the at least one suggestion, a suggestion to unlock electronic lock <NUM> on entrance door <NUM> of the building.

With this, when item <NUM> of an urgent delivery arrives while user <NUM> is not cooking, it is possible to effectively present the user with a suggestion of an action necessary for receiving item <NUM>. In such a case, user <NUM> is less likely to select, as the next action, request redelivery of item <NUM>, and thus, it is possible to more effectively present user <NUM> with information which assists user <NUM> in determining the next action.

In information processing system <NUM> according to the present embodiment, in (c), when the user state indicates that user <NUM> is not cooking and the urgent information indicates that the delivery is urgent, information processing server <NUM> can obtain, as the at least one suggestion, a suggestion to receive item <NUM> and a suggestion to request redelivery of item <NUM>.

With this, when item <NUM> of an urgent delivery arrives while user <NUM> is not cooking, it is possible to present a suggestion to receive item <NUM> and a suggestion to request redelivery of item <NUM>, from which user <NUM> can select the next action. That is to say, with reference to the content information of item <NUM>, user <NUM> can determine, for each item <NUM>, whether to receive item <NUM> or to request redelivery of item <NUM>. In other words, information processing server <NUM> can more effectively present user <NUM> with information which assists user <NUM> in determining the next action.

Note that although the content information of item <NUM> according to the present embodiment includes the urgent information, the content information of item <NUM> is not limited to this. For example, the content information of item <NUM> may include the type of item <NUM>. For example, the type of item <NUM> is fresh food, frozen food, or the like. In such a case, at least one suggestion is obtained based on the user state and the type of item <NUM>.

Note that the content information of item <NUM> in the present embodiment includes an image of item <NUM>; however, a product name of item <NUM> may be included instead of an image of item <NUM>. The content information of item <NUM> may include both an image of item <NUM> and a product name of item <NUM>.

Note that the screens and data illustrated in the present embodiment are mere examples, and the present disclosure is not limited to these examples. For example, the display regions in which the three pieces of information are displayed need not be identical in size and shape. The combinations of (i) a user state and an urgent flag and (ii) a first suggestion and a second suggestion, illustrated in <FIG>, are a mere example, and the present disclosure is not limited to this example. First suggestions and second suggestions different from those in <FIG> may be associated with user states and urgent flags.

For example, as illustrated in <FIG>, Unlock may be used as the first suggestion and Receive Later as the second suggestion. For example, as illustrated in <FIG>, Cooker OFF may be used as the first suggestion and Receive Later as the second suggestion. For example, as illustrated in <FIG>, Receive Now may be used as the first suggestion and Select Redelivery Time as the second suggestion. For example, as illustrated in <FIG>, Unlock & Cooker OFF may be used as the first suggestion and Receive Later as the second suggestion.

Note that the user states and the values of the urgent flag that lead to the screen examples in <FIG> are not particularly limited, and any user state and any value of the urgent flag may lead to the screen examples in <FIG>.

Note that in the present embodiment, first data 801a, second data 802a, and third data 803a are referred to, however; the present disclosure is not limited to this. For example, first data 801a need not be referred to. In such a case, whether or not the user is currently cooking may be directly determined based on the ON/OFF signal of cooker <NUM>, for example. Further, second data 802a need not be referred to. In such a case, the content information of item <NUM> may be directly obtained from a camera image, for example. Further, third data 803a need not be referred to. In such a case, third data 803a may be hard coded.

Note that in the present embodiment, the processing order of the flow chart in <FIG> is a mere example, and the present disclosure is not limited to this example. For example, Steps S201 and S202 may be performed before Steps S101 and <NUM>.

After the service providing system is used for a certain time period and after Step S303 is finished, a selection screen may be displayed for the user to select the user's preferred display mode for display <NUM> in Step S302, out of a display mode in which plural pieces of information are sequentially displayed in the display regions and a display mode in which plural pieces of information are simultaneously displayed in the display regions. The display mode for Step S302 which is to be applied next time the service is provided is determined based on the display mode selected by the user.

Note that the user may be given a choice among the first to fifth screens illustrated in <FIG>. For example, the initial setting may be the first screen, and after the processing of the flow chart illustrated in <FIG> is finished, display <NUM> may display, to the user, the first to fifth screens and a message prompting the user to select a screen which the user wishes to be displayed from next time onward. From next time onward, suggestions are displayed to the user based on the screen selected by the user. For example, the first to fifth screens are displayed in a temporal order. Further, when processing is performed a certain number of times on the screen which the user has selected, display <NUM> may display again the first to fifth screens and a message prompting the user to select a screen which the user wishes to be displayed from next time onward. Examples of the certain number of times include <NUM>, <NUM>, and <NUM>. When the user is repeatedly prompted to select a screen for every certain number of times and the user successively selects the same screen, the user need not be prompted to select a screen after that.

Next, Embodiment <NUM> will be described. The present embodiment is not according to the claimed invention and describes a service providing system that provides a service including provision of information on a screen to be displayed on a display terminal.

<FIG> illustrates an overview of a service providing system according to Embodiment <NUM>.

Group <NUM> may be, for example, a company, an organization, or a household of any size. Group <NUM> includes device A and device B that are included in devices <NUM>, and home gateway <NUM>. Device A is, for example, cooker <NUM> according to Embodiment <NUM>. Devices <NUM> include devices connectable to the Internet (e.g., a smartphone, PC, and TV) and devices unconnectable to the Internet by themselves (e.g., an illumination and a washing machine). Devices <NUM> may include devices that are unconnectable to the Internet by themselves but are connectable to the Internet via home gateway <NUM>. Moreover, group <NUM> includes users 990A of devices <NUM>.

Data center operating company <NUM> includes cloud server <NUM>. Cloud server <NUM> is a virtual server that operates in cooperation with various devices via the Internet. For example, cloud server <NUM> is information processing server <NUM> according to Embodiment <NUM>. Cloud server <NUM> mainly manages large data (big data) that is hard to handle with ordinary database management tools or the like. Data center operating company <NUM> manages data and cloud server <NUM> and, for example, operates a data center for the management. Service provided by data center operating company <NUM> will be specifically described later. In this case, data center operating company <NUM> is not limited to companies that are only involved in data management and the operation etc. of cloud server <NUM>. For example, if a device manufacturer that develops and manufactures one of devices <NUM> also manages data and cloud server <NUM>, the device manufacturer is equivalent to data center operating company <NUM> (<FIG>). Data center operating company <NUM> is not limited to a single company. For example, if a device manufacturer and another management company are involved in data management and the operation of cloud server <NUM> in a collaborative or shared manner, at least one of the device manufacturer and the management company is equivalent to data center operating company <NUM> (<FIG>).

Service provider <NUM> possesses server <NUM>. In this configuration, server <NUM> includes, for example, memory in a personal PC regardless of the size. The service provider may not possess server <NUM>.

In this service, home gateway <NUM> is not mandatory. For example, if cloud server <NUM> totally manages data, home gateway <NUM> is not necessary. There are also cases where devices unconnectable to the Internet by themselves are not provided, as in the case where all the home devices are connected to the Internet.

The flow of information in the service will be described below.

First, device A or device B of group <NUM> transmits information obtained in each device to cloud server <NUM> of data center operating company <NUM>. Cloud server <NUM> accumulates information on device A or device B ((a) of <FIG>). The accumulated information indicates, for example, the operating condition, the operation date and time, the operation mode, and the position of each device <NUM>. For example, the information includes but is not limited to the television viewing history, information on the recording schedule of a recorder, the operating date and time of a washing machine, the amount of laundry, the date and time of opening/closing of a refrigerator, the number of times of opening/closing of a refrigerator, and the amount of foods in a refrigerator. Any kinds of information available from all the devices can be used. Information may be directly provided to cloud server <NUM> from devices <NUM> via the Internet. Alternatively, information may be temporarily accumulated in home gateway <NUM> from devices <NUM> and then provided to cloud server <NUM> from home gateway <NUM>.

Next, cloud server <NUM> of data center operating company <NUM> provides service provider <NUM> with the accumulated information in constant units. In this case, the constant units may be units in which the data center operating company can organize and provide the accumulated information to service provider <NUM> or units requested by service provider <NUM>. The units need not be constant units. The amount of information provided may vary depending on the circumstances. Where necessary, the information is stored in server <NUM> provided in service provider <NUM> ((b) in <FIG>). Then, service provider <NUM> organizes the information suitably for user service and then provides the information to users. The users whom the information is provided to may be users 990A of devices <NUM> or external users 990B. The service may be directly provided to users from the service provider, for example ((e) and (f) of <FIG>). The service may be provided to users after passing through, for example, cloud server <NUM> of data center operating company <NUM> again ((c) and (d) of <FIG>). Cloud server <NUM> of data center operating company <NUM> may organize the information suitably for user service and then provide the information to service provider <NUM>.

User 990A and user 990B may be different from each other or identical to each other.

Techniques described in the foregoing embodiments can be implemented by, for example, the following types of cloud service. However, types of cloud service for implementing the techniques described in the embodiments are not limited to the following types:.

<FIG> illustrates service type <NUM> (company-owned data center). In this type, service provider <NUM> obtains information from group <NUM> and provides service to users. In this type, service provider <NUM> has the function of the data center operating company. Specifically, the service provider includes cloud server <NUM> that manages big data. Thus, data center operating companies are not present.

In this type, service provider <NUM> operates and manages data center <NUM> (cloud server <NUM>). Moreover, service provider <NUM> manages OS <NUM> and application <NUM>. Service provider <NUM> provides service <NUM> by using OS <NUM> and application <NUM> that are managed by service provider <NUM>.

<FIG> illustrates service type <NUM> (IaaS-based). IaaS stands for Infrastructure as a Service, which is a cloud service provision model for providing, as service via the Internet, an infrastructure for constructing and operating a computer system.

In this type, data center operating company <NUM> operates and manages data center <NUM> (cloud server <NUM>). Moreover, service provider <NUM> manages OS <NUM> and application <NUM>. Service provider <NUM> provides service <NUM> by using OS <NUM> and application <NUM> that are managed by service provider <NUM>.

<FIG> illustrates service type <NUM> (PaaS-based). PaaS stands for Platform as a Service, which is a cloud service provision model for providing, as service via the Internet, a platform for constructing and operating software.

In this type, data center operating company <NUM> manages OS <NUM> and operates and manages data center <NUM> (cloud server <NUM>). Moreover, service provider <NUM> manages application <NUM>. Service provider <NUM> provides service <NUM> by using OS <NUM> managed by the data center operating company and application <NUM> managed by service provider <NUM>.

<FIG> illustrates service type <NUM> (SaaS-based). SaaS stands for Software as a Service. For example, SaaS is a cloud service provision model that enables a company/individual (user) who does not have a data center (cloud server) to use an application provided by a platform provider having a data server (cloud server) via a network such as the Internet.

In this type, data center operating company <NUM> manages application <NUM>, manages OS <NUM>, and operates and manages data center <NUM> (cloud server <NUM>). Service provider <NUM> provides service <NUM> by using OS <NUM> and application <NUM> that are managed by data center operating company <NUM>.

In all of the types, service is provided by service provider <NUM>. For example, the service provider or the data center operating company may develop its own OS, application, or the database of big data etc., or outsource the development to a third party.

For example, an aspect of the present disclosure may be a display control method which includes, as steps, not only processing performed by such an information processing system, but also processing performed by a processor included in the information processing system. Further, an aspect of the present disclosure may be a computer program which causes a computer to perform each characteristic step included in a display control method. Furthermore, an aspect of the present disclosure may be a non-transitory computer-readable recording medium having such a computer program recorded thereon.

Such a program causes a computer to (a) obtain, using a sensor, a user state indicating whether a user is currently cooking using a cooker, when an image of a delivery person delivering an item to a building in which the user is present is captured by a camera provided at an entrance door of the building; (b) obtain content information of the item; (c) obtain at least one suggestion associated with the user state and the content information of the item; and (d) cause a display terminal provided in the building to display the image of the delivery person, the content information of the item, and the at least one suggestion.

In the present disclosure, part or all of the units and devices, or part or all of the functional blocks in the block diagram illustrated in <FIG> may be implemented by one or more electric circuits including a semiconductor device, a semiconductor integrated circuit (IC), or a large scale integration (LSI) circuit. LSI or IC may be integrated in a single chip, or may be configured by combining a plurality of chips. For example, functional blocks other than memory elements may be integrated in a single chip. Although the names used here are LSI and IC, but the integrated circuit may also be called a system LSI circuit, a very LSI circuit (VLSI), or an ultra LSI (ULSI) circuit depending on the degree of integration. A field programmable gate array (FPGA) programed after manufacturing of the LSI circuit or a reconfigurable logic device which is reconfigurable in connection or settings of circuit cells inside the LSI circuit may be used for the same purpose.

Moreover, some or all of the functions or operations of the units, devices, or part of the devices can be performed through software processing. In such a case, software is recorded on a non-transitory recording medium such as one or more ROMs, optical discs, or hard disk drives, and the software, when executed by a processor, causes the processor and a peripheral device to perform particular functions of the software. The system or device may include one or more non-transitory recording media having software recorded thereon, a processor, and a necessary hardware device, such as an interface.

Claim 1:
A display control method performed using a system including an information processing server (<NUM>), the display control method comprising:
(a) obtaining, by the information processing server (<NUM>), using a sensor (<NUM>), a user state indicating whether a user (<NUM>) is currently cooking using a cooker (<NUM>), when an image of a delivery person (<NUM>) delivering an item (<NUM>) to a building in which the user (<NUM>) is present is captured by a camera (<NUM>) provided at an entrance door (<NUM>) of the building;
(b) obtaining, by the information processing server (<NUM>), content information of the item (<NUM>);
(c) obtaining, by the information processing server (<NUM>), at least one suggestion associated with the user state and the content information of the item (<NUM>); and
(d) causing, by the information processing server (<NUM>), a display terminal (<NUM>) provided in the building to display the image of the delivery person (<NUM>), the content information of the item (<NUM>), and the at least one suggestion.