Patent Publication Number: US-2015086074-A1

Title: Information processing device, information processing method, and program

Description:
TECHNICAL FIELD 
     The present invention relates to an information processing device, an information processing method, and a program. 
     BACKGROUND ART 
     In recent years, techniques for specifying a person shown in a captured image have been developed. For example, a technique is disclosed in which a facial region shown in a captured image is checked against a face image prepared in advance and a person shown in the captured image is specified on the basis of the checking result (for example, see Patent Literature 1). According to the technique, a door is opened/closed in the case where the specifying of the person has been succeeded, and hence, enhanced security can be achieved. 
     The technique for specifying a person shown in a captured image is applied not only for enhancing security, but is also applied to various scenes. 
     CITATION LIST 
     Patent Literature 
     Patent Literature 1: JP 2009-3659A 
     SUMMARY OF INVENTION 
     Technical Problem 
     However, in a case where it is attempted to specify a user from a captured image, the user may not necessarily be specified successfully. For example, a situation is assumed that the user may not face an imaging device. In such a situation, a feature portion of the user is not shown in the captured image, and hence, the specification of the user may fail. Accordingly, it is desired to achieve a technique for easily specifying the user from the captured image. 
     Solution to Problem 
     According to the present disclosure, there is provided an information processing device including an image acquisition unit configured to acquire a captured image, a parameter acquisition unit configured to acquire a parameter associated with a user of a portable terminal, a specification unit configured to specify the user from the captured image on the basis of the parameter, and a display control unit configured to perform control in a manner that, in a case where the user is not specified by the specification unit, information indicating a movement of the user necessary to specify the user is displayed in the portable terminal. 
     According to the present disclosure, there is provided an information processing method including acquiring a captured image, acquiring a parameter associated with a user of a portable terminal, specifying the user from one or more subjects shown in the captured image on the basis of the parameter, and performing control in a manner that, in a case where the user is not specified, information indicating a movement of the user necessary to specify the user is displayed in the portable terminal. 
     According to the present disclosure, there is provided a program for causing a computer to function as an information processing device including an image acquisition unit configured to acquire a captured image, a parameter acquisition unit configured to acquire a parameter associated with a user of a portable terminal, a specification unit configured to specify the user from one or more subjects shown in the captured image on the basis of the parameter, and a display control unit configured to perform control in a manner that, in a case where the user is not specified by the specification unit, information indicating a movement of the user necessary to specify the user is displayed in the portable terminal. 
     Advantageous Effects of Invention 
     According to the present disclosure as described above, the user can be easily specified from the captured image. 
    
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         FIG. 1  is a diagram illustrating a brief overview in accordance with a present embodiment. 
         FIG. 2  is a diagram illustrating a configuration of an information processing system in accordance with the present embodiment. 
         FIG. 3  is a block diagram illustrating a functional configuration of an information processing device in accordance with the embodiment. 
         FIG. 4  is a diagram illustrating a brief overview of a user specification function of an information processing device. 
         FIG. 5  is a diagram illustrating a display example of a portable terminal when the user specification fails. 
         FIG. 6  is a diagram illustrating a display example of a portable terminal when the user specification succeeds. 
         FIG. 7  is a diagram illustrating a display example of a portable terminal when the user specification fails. 
         FIG. 8  is a diagram illustrating a display example of a portable terminal when the user specification fails. 
         FIG. 9  is a diagram illustrating a display example of a portable terminal when the user specification fails. 
         FIG. 10  is a diagram illustrating a brief overview of a composition decision function of the information processing device. 
         FIG. 11  is a diagram illustrating a display image at the time of setting a first mode. 
         FIG. 12  is a diagram illustrating a display image at the time of setting a second mode. 
         FIG. 13  is a diagram illustrating a variation of a technique of determining a display image. 
         FIG. 14  is a diagram illustrating an example of the determined display image. 
         FIG. 15  is a flowchart illustrating a flow of composition decision operations of the information processing device. 
         FIG. 16  is a diagram illustrating an example of a depiction change function of the information processing device. 
         FIG. 17  is a diagram illustrating another example of a depiction change function of the information processing device. 
         FIG. 18  is a diagram illustrating another example of a depiction change function of the information processing device. 
         FIG. 19  is a diagram illustrating another example of a depiction change function of the information processing device. 
         FIG. 20  is a diagram illustrating another example of a depiction change function of the information processing device. 
         FIG. 21  is a diagram illustrating another example of a depiction change function of the information processing device. 
         FIG. 22  is a diagram illustrating another example of a depiction change function of the information processing device. 
         FIG. 23  is a flowchart illustrating a flow of depiction change operations of the information processing device. 
     
    
    
     DESCRIPTION OF EMBODIMENTS 
     Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the appended drawings. Note that, in this specification and the appended drawings, structural elements that have substantially the same function and structure are denoted with the same reference numerals, and repeated explanation of these structural elements is omitted. 
     Further, in this specification and the drawings, each of a plurality of structural elements having substantially the same function is distinguished by affixing a different letter to the same reference numeral in some cases. However, when there is no particular necessity to distinguish between structural elements having substantially the same function, they are denoted by the same reference numeral. 
     First, “Description of Embodiments” will be described in accordance with the order listed below. 
     1. User specification function
 
2. Composition decision function
 
3. Depiction change function
 
     4. Conclusion 
     Functions of the information processing device according to the present embodiment are broadly classified into a user specification function, a composition specification function, and a depiction change function. First, before describing each of the functions, a condition that is a premise of each function will be described. 
       FIG. 1  is a diagram illustrating a brief overview of the present embodiment. As shown in  FIG. 1 , a user U having a portable terminal  20  is present in a real space. The user U may be present anywhere in the real space. In addition, a function of displaying an image is installed in the portable terminal  20 , and it is possible for the user U to view a screen displayed by the portable terminal  20 . The user U is captured by an imaging device  30  provided to be separate from the portable terminal  20 . A position in which the imaging device  30  is to be provided is not particularly limited. For example, the imaging device may be provided in a building such as in a monitoring camera, or may be provided in a mobile object such as a vehicle. In addition, a kind of the imaging device  30  is not particularly limited, but may be an infrared camera or the like, for example. 
       FIG. 2  is a diagram illustrating a configuration of an information processing system in accordance with the present embodiment. As shown in  FIG. 2 , the information processing system  1  includes an information processing device  10 , a portable terminal  20 , and an imaging device  30 . Although the information processing device  10  and the portable terminal  20  are connected via a network  40  in  FIG. 2 , the number of the portable terminal  20  connected to the network  40  may not be one. For example, a plurality of the portable terminals  20  may be present, and each of the portable terminals  20  may be connected to the network  40 . 
     In addition, as shown in  FIG. 2 , the information processing device  10  is connected to the imaging device  30 . Although one imaging device  30  is connected to the information processing device  10  in the example shown in  FIG. 2 , the number of the imaging device  30  may not be one. For example, a plurality of the imaging devices  30  may be present, and each of the imaging devices  30  may be connected to the information processing device  10 . 
     In addition, although the information processing device  10  is configured to be separate from the portable terminal  20  and the imaging device  30  in the example shown in  FIG. 2 , the information processing device may be embedded in the portable terminal  20 , or may be embedded in the imaging device  30 . A captured image captured by the imaging device  30  is provided to the portable terminal  20  via the information processing device  10 , and it is possible for the portable terminal  20  to display the captured image. The information processing device  10  may use the captured image provided from the imaging device  30  to exert the user specification function, the composition decision function, and the depiction change function. 
       FIG. 3  is a block diagram illustrating functional configuration of the information processing device  10  in accordance with the present embodiment. As shown in  FIG. 3 , the information processing device  10  includes a processing control unit  100 , a display control unit  160 , and a storage unit  50 . In addition, the processing control unit  100  has an image acquisition unit  111 , a parameter acquisition unit  112 , a candidate extraction unit  113 , a specification unit  114 , and an authentication unit  115 . The image acquisition unit  111 , the parameter acquisition unit  112 , the candidate extraction unit  113 , the specification unit  114 , and the authentication unit  115  are blocks mainly associated with the user specification function of the information processing device  10 . 
     In addition, the processing control unit  100  has a recognition unit  121 , a condition decision unit  122 , an information addition unit  123 , a mode setting unit  124 , a composition decision unit  125 , and an image determination unit  126 . The processing control unit  100  is, and the recognition unit  121 , the condition decision unit  122 , the information addition unit  123 , the mode setting unit  124 , the composition decision unit  125 , and the image determination unit  126  are blocks mainly associated with the composition decision function of the information processing device  10 . 
     In addition, the processing control unit  100  has a change detection unit  131  and a depiction change unit  132 . The change detection unit  131  and the depiction change unit  132  are blocks mainly associated with the depiction change function of the information processing device  10 . 
     The processing control unit  100  and the display control unit  160  correspond to processors such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor). The processing control unit  100  and the display control unit  160  exert various functions of the processing control unit  100  and the display control unit  160  by executing a program stored in the storage unit  50  or another storage medium. 
     The storage unit  50  uses a storage medium such as a semiconductor memory or a hard disk to store programs and data for processes to be carried out by the processing control unit  100  and the display control unit  160 . For example, the storage unit  50  stores a feature amount dictionary used to recognize an object. Further, the storage unit  50  may store a recognition result that is generated as a result of the object recognition. Although the storage unit  50  is configured to be embedded in the information processing device  10  in the example shown in  FIG. 3 , the storage unit  50  may be configured to be separate from the information processing device  10 . 
     Premises of the respective functions have been described. Hereinafter, each of the functions will be described in order. 
     1. User Specification Function 
     First, the user specification function of the information processing device  10  will be described.  FIG. 4  is a diagram illustrating a brief overview of the user specification function of the information processing device  10 . The image acquisition unit  111  first acquires a captured image captured by the imaging device  30 . In the example shown in  FIG. 4 , as an example of the captured image, a captured image Img in which a real space shown in  FIG. 1  is captured is acquired by the image acquisition unit  111 . The captured image acquired by the image acquisition unit  111  is controlled to be displayed in the portable terminal  20  by the display control unit  160 . 
     According to the user specification function, a user is specified from a subject shown in the captured image acquired by the image acquisition unit  111 . The purpose of the user specification is not particularly limited. However, it is assumed herein that a network service is provided by the information processing device  10  and the user specification is carried out so as to allow the user U to log into the network service via the portable terminal  20 . 
     In addition, the kind of the network service is not particularly limited. However, a service allowing a user to write a message or to read the message written by the user or another user using a portable terminal in which logging into the network service is completed is assumed herein. Although the timing for specifying the user is not particularly limited, a timing at which an operation for the user to request the user specification is performed on the portable terminal  20  may be used. The operation for requesting the user specification may be an operation for request to log into the network service. 
     In order to specify the user from the subject shown in the captured image, the parameter acquisition unit  112  acquires parameters associated with the user to be specified. Although the timing for the parameter acquisition unit  112  to acquire the parameters is not particularly limited, for example, the parameters may be acquired in advance from the portable terminal  20  of the user, or may be acquired from the portable terminal  20  when the user is specified. 
     The kind of the parameter to be acquired by the parameter acquisition unit  112  is not particularly limited. For example, as shown in  FIG. 4 , the parameter acquisition unit  112  may acquire a face image of the user as the parameter P 1 , and may acquire sensor data representing a movement of the user as the parameter P 2 . A device capturing the face image of the user is not particularly limited. For example, the face image of the user may be a face image captured by the portable terminal  20 , or may be a face image captured by another device. 
     In addition, a sensor detecting the movement of the user is not particularly limited. The sensor detecting the movement of the user may be an acceleration sensor, a sensor receiving GPS signals, and a sensor receiving radio waves transmitted from a Wi-Fi base station. In addition, the sensor detecting the movement of the user may be embedded in the portable terminal  20 , and may be mounted on a body of the user. 
     Before the user is specified by the specification unit  114 , a candidate for the user may be specified by the candidate extraction unit  113  (step S 11 ). For example, when coordinates on the captured image displayed in the portable terminal  20  by the user operation input to the portable terminal  20  (e.g., a touch operation of the touch panel) are designated, the parameter acquisition unit  112  acquires the coordinates, and the candidate extraction unit  113  may specify a subject selected by the coordinates (e.g., the subject present on the coordinates) as the candidate for the user. By specifying the candidate for the user, a load necessary to specify the user is reduced. 
     The specification unit  114  tries to specify the user from the captured image based on the parameter acquired by the parameter acquisition unit  112  (step S 12 ). For example, when the face image of the user is acquired as the parameter P 1  by the parameter acquisition unit  112 , the specification unit  114  may specify the subject extracted based on checking of a facial region of the subject shown in the captured image acquired by the image acquisition unit  111  against the face image as the user. For example, the specification unit  114  may specify the subject having the face image of the user and the facial region of the similarity exceeding a threshold value as the user. 
     In addition, for example, when sensor data representing the movement of the user is acquired as the parameter P 2  by the parameter acquisition unit  112 , the specification unit  114  may specify the subject extracted based on a checking of the movement of the subject detected from the captured image acquired by the image acquisition unit  111  against the sensor data as the user. For example, the specification unit  114  may specify the subject in which the movement of the user in the captured image and the sensor data of the similarity exceeding a threshold value are detected as the user. In addition, when the candidate for the user is specified, the specification unit  114  may specify the user based on the parameter from the candidate for the user. 
     As a result of the specification unit  114  trying to specify the user, it is also assumed that the user is not specified. For example, a case in which the similarity does not exceed the threshold value corresponds to the situation assumed above. Accordingly, when the user is not specified by the specification unit  114 , the display control unit  160  controls the portable terminal  20  to display information indicating the movement of the user necessary to specify the user in the portable terminal  20 . 
     In accordance with the control of the display control unit  160 , the information indicating the movement of the user is displayed in the portable terminal  20  (step S 13 ). 
       FIG. 5  is a diagram illustrating a display example of the portable terminal  20  when the user specification fails. For example, when the user is not specified by the specification unit  114 , the display control unit  160  may control the portable terminal  20  such that the information prompting the user to turn his or her face toward the imaging device is displayed in the portable terminal  20 . In  FIG. 5 , the example in which the imaging device  30  is assumed to be a monitoring camera and the display control unit  160  controls the portable terminal  20  such that a message M 1  indicating “Please turn your face toward the monitoring camera” is displayed in the portable terminal  20  is shown. 
     A description will be made with reference back to  FIG. 4 . When the user is specified by the specification unit  114 , before the flow proceeds to step S 15 , the authentication unit  115  may perform the authentication process on the portable terminal  20  specified by the specification unit  114  (step S 14 ). In particular, for example, as shown in FIG,  4 , when a password is input by a user operation input to the portable terminal  20 , the authentication unit  115  may determine success and failure by determining whether the input password matches a password that is registered in advance. When the authentication is successful, the flow proceeds to step S 15 . When the authentication fails, the authentication process may be performed again instead of proceeding to step S 15 . 
     When the flow proceeds to step S 15 , for example, logging into the network service is completed (step S 15 ) and the display control unit  160  controls the portable terminal  20  such that the captured image and the additional information are displayed by the portable terminal  20 . In accordance with the control of the display control unit  160 , the captured image and the additional information are displayed in the portable terminal  20  (step S 16 ). Although the additional information is not particularly limited, for example, the additional information may be a message written by the user himself/herself or another user. 
       FIG. 6  is a diagram illustrating a display example of the portable terminal  20  when the user specification is successful. For example, when the user is specified by the specification unit  114 , the display control unit  160  controls the portable terminal  20  such that the captured image and the additional information are displayed. An example in which the display control unit  160  controls the portable terminal  20  such that a message written by another user is displayed as the additional information is shown in  FIG. 6 . 
     In addition, the information prompting the user to turn his or her face toward the imaging device is not limited to the example shown in  FIG. 5 .  FIG. 7  is a diagram illustrating a display example of the portable terminal  20  when the user specification fails. The display control unit  160  may control the portable terminal  20  such that information indicating the movement of the user necessary to specify the user is displayed in the portable terminal  20  based on a position of the candidate for the user extracted by the candidate extraction unit  113  and a position of the imaging device  30  that is set in advance. 
     For example, as shown in  FIG. 7 , the display control unit  160  may control the portable terminal  20  such that the portable terminal  20  displays a straight line in a direction that the candidate for the user faces and a straight line in a direction in which the imaging device  30  is present using the position of the candidate for the user as a reference. In addition, as shown in  FIG. 7 , the display control unit  160  may control the portable terminal  20  such that an arrow representing a rotational direction from the straight line in the direction that the candidate for the user faces to the straight line in the direction in which the imaging device  30  is present is displayed in the portable terminal  20 . In addition, as shown in  FIG. 7 , the display control unit  160  may control the portable terminal  20  such that a message M 2  of “Turn Left” is displayed as the message representing the rotational direction that the user has seen in the portable terminal  20 . 
     In addition, when the user is not specified by the specification unit  114 , a technique of specifying the user may be switched.  FIG. 8  is a diagram illustrating a display example of the portable terminal  20  when the user specification fails. When the user is not specified by the specification unit  114 , the display control unit  160  may control the portable terminal  20  such that information indicating that the user is specified by another technique is displayed in the portable terminal  20 . For example, the display control unit  160  may control the portable terminal  20  such that a message M 3  of “a technique of specifying the user is switched” is displayed as the information indicating that the user is specified by another technique in the portable terminal  20 . The specification unit  114  may specify the subject extracted by another specification technique as the user. 
     For example, the specification unit  114  tries to specify the subject extracted based on checking of the face region of the subject shown in the captured image against the face image of the user as the user. In this case, when the user is not specified, the specification unit  114  may try to specify the subject extracted based on checking of the movement of the subject detected by the captured image against sensor data as the user. In addition, the user specification may be tried in reverse order. 
     In addition, when the user is not specified by the specification unit  114 , the user may be prompted to cause a difference between movements of another subject shown in the captured image and the user to be clear. Accordingly, when a subject moving out of the defined normal range is detected from the captured image, it is possible for the specification unit  114  to specify the subject as the user. 
       FIG. 9  is a diagram illustrating a display example of the portable terminal  20  when the user specification is failed. For example, when the user is not specified by the specification unit  114 , the display control unit  160  may control the portable terminal  20  such that information prompting the user to move the portable terminal  20  is displayed as information indicating the movement of the user necessary to specify the user in the portable terminal  20 . For example, as shown in  FIG. 9 , the display control unit  160  may control the portable terminal  20  such that a message M 4  of “Please move your portable terminal” is displayed as the information prompting the user to move the portable terminal  20  in the portable terminal  20 . 
     As described above, according to the user specification function of the information processing device  10 , user specification is tried from the captured image based on the parameter associated with the user of the portable terminal, and the display control unit  160  controls the portable terminal  20  such that information indicating the movement of the user necessary to specify the user is displayed in the portable terminal  20  when the user is not specified. Therefore, the user is expected to be easily specified from the captured image. 
     2. Composition Decision Function 
     Next, the composition decision function of the information processing device  10  will be described.  FIG. 10  is a diagram illustrating a brief overview of the composition decision function of the information processing device  10 . As described above, when the capture image is captured by the imaging device  30 , the captured image is displayed in the portable terminal  20  of the user. It is possible for the user to view the captured image displayed in the portable terminal  20 . A scene in which the user moves toward the destination while viewing the captured image is assumed herein. Decision on a preferred composition as the captured image viewed by the user will be described. 
     First, information indicating the destination of the user is assumed. The information indicating the destination of the user may be input by the user operation input to the portable terminal  20 , and may be set by an application run by the portable terminal  20 . For example, the information indicating the destination of the user is depicted at a position in the real space. The information indicating the destination of the user is acquired by the information processing device  10 . The composition decision unit  125  decides the composition based on the information indicating the destination of the user and the recognition result of the user based on the captured image acquired by the image acquisition unit  111 . The composition is a diagram illustrating an ideal position of an object in the image (or an ideal direction of the object). 
     The recognition result of the user includes at least the user position in the real space and the travelling direction of the user. For example, the recognition result of the user is acquired by the recognition unit  121 . In other words, the recognition unit  121  obtains the recognition result by recognizing the user based on the captured image. In the example shown in  FIG. 10 , the user position in the real space is denoted as Pu, the travelling direction of the user is denoted as V 1 , and the direction of the destination of the user using the user position Pu as a reference is denoted as V 2 . 
     For example, when the user position is recognized as the recognition result, the composition decision unit  125  may decide the composition based on the direction of the destination of the user using the user position as a reference. For example, in the example shown in  FIG. 10 , the composition decision unit  125  may decide the composition based on the direction V 2  of the destination of the user using the user position Pu as the reference. For example, when the composition is decided based on V 2 , the composition decision unit  125  may decide the image in which a region in the direction V 2  using the user position Pu as the reference is included to be larger than a region in a direction opposite to the direction V 2  using the user position Pu as the reference as the composition. An example of the composition as decided above is shown as K 2  in  FIG. 10 . The point C 2  indicates a middle point of the composition K 2 . 
     In addition, for example, when the travelling direction of the user is recognized as the recognition result, the composition decision unit  125  may decide the composition based on the travelling direction of the user. For example, in the example shown in  FIG. 10 , the composition decision unit  125  may decide the composition based on the travelling direction V 1  of the user. For example, when the composition is decided based on V 1 , the composition decision unit  125  may decide the image in which a region in the direction V 1  using the user position Pu as the reference is included to be larger than a region in a direction opposite to the direction V 1  using the user position Pu as the reference as the composition. An example of the composition as decided above is shown as K 1  in FIG,  10 . The point C 1  indicates a middle point of the composition K 1 . 
     The composition decision unit  125  may uniformly decide the composition based on the travelling direction V 1  of the user, may uniformly decide the composition based on the direction V 2  of the destination of the user using the user position Pu as the reference, or may selectively use any of the decision techniques mentioned above. In other words, when a mode in which the composition is decided based on the travelling direction V 1  of the user is a first mode and a mode in which the composition is decided based on the direction V 2  of the destination of the user using the user position Pu as the reference is a second mode, the mode setting unit  124  may set any of these two modes. In this case, the composition decision unit  125  may decide the composition in accordance with the mode set by the mode setting unit  124 . The mode may be set based on the user operation input to the portable terminal  20 , or may be set by an application run by the portable terminal  20 . 
     The image determination unit  126  determines the display image based on the composition decided by the composition decision unit  125 . For example, the image determination unit  126  may determine an image in conformity with the composition decided by the composition decision unit  125  as the display image, or may determine an image closest to the composition decided by the composition decision unit  125  within the selectable range as the display image. The display control unit  160  controls the portable terminal  20  such that the display image determined by the image determination unit  126  is displayed in the portable terminal  20 . 
       FIG. 11  is a diagram illustrating a display image at the time of setting the first mode. For example, when the captured image Img is acquired as shown in  FIG. 10  and the first mode is set by the mode setting unit  124 , the composition K 1  is decided based on the travelling direction V 1  of the user. In this case, for example, as shown in  FIG. 11 , the image determination unit  126  may determine an image in conformity with the composition K 1  as the display image D 1 . In addition, the composition decision unit  125  may decide the composition in which the user is present in the middle. As shown in  FIG. 11 , the composition in which the user is present at a position spaced a predetermined distance dl from the middle may be decided. 
     In addition, a current capturing condition may be decided. In other words, the condition decision unit  122  may decide whether or not the condition for capturing the user is prepared, and the information addition unit  123  may add information indicating that the condition is not prepared to the display image when the condition decision unit  122  decides that the condition is not prepared. For example, when a region capable of being captured is decided by the condition decision unit  122 , the information addition unit  123  may add an object indicating a boundary of the region capable of being captured to the display image. 
     In addition, when the boundary of the region capable of being captured in the travelling direction V 1  of the user using the user position Pu as the reference is decided by the condition decision unit  122 , the information addition unit  123  may add an object indicating the boundary of the region capable of being captured to the display image. Since the boundary of the region capable of being captured in the travelling direction V 1  of the user using the user position Pu as the reference has been decided in  FIG. 11 , the object indicating the boundary of the region capable of being captured is added to the display image D 1  as A 1 . Alternatively, when the condition decision unit  122  decides that a distance between the user position Pu and the boundary of the region capable of being captured is shorter than a threshold, the information addition unit  123  may add the object indicating the boundary of the region capable of being captured to the display image. 
     In addition, when the condition decision unit  122  decides that the brightness of the captured image is less than a threshold, the information addition unit  123  may add an object indicating deterioration of the capturing condition to the display image. Although prompting the user to change the travelling direction is possible by virtue of such a process, the information provided to the user is not limited to the added object. For example, the information addition unit  123  may control the portable terminal  20  to vibrate, and may control the portable terminal  20  to generate an alarm tone. 
       FIG. 12  is a diagram illustrating a display image at the time of setting the second mode. For example, when the captured image Img is acquired as shown in  FIG. 10  and the second mode is set by the mode setting unit  124 , the composition K 2  is decided based on the direction V 2  of the destination of the user using the user position Pu as the reference. In this case, for example, as shown in  FIG. 12 , the image determination unit  126  may determine an image in conformity with the composition K 2  as the display image D 2 . In addition, although the composition decision unit  125  decides the composition in which the user is present in the middle, the composition decision unit may decide the composition in which the user is present at a position spaced a predetermined distance d2 from the middle as shown in  FIG. 12 . 
     So far, the image determination unit  126  has used the technique of determining the display image by cutting out a portion of the captured image captured by the imaging device  30  based on the composition decided by the composition decision unit  125 . When the imaging device  30  is one fixed camera, only the technique mentioned above is employed. However, when the imaging device  30  is a direction adjustable camera or an angle-of-view adjustable camera, or when the imaging device  30  is provided in a plural number, other techniques may be employed. 
     In this case, the fixed camera means a camera in which the capturing direction or the angle of view is not adjustable. The direction adjustable camera means a movable camera, and may be a camera having a pan function or a camera having a tilt function. The angle-of-view adjustable camera means a camera capable of changing the angle of view.  FIG. 13  is a diagram illustrating a variation of a technique of determining the display image. As shown in  FIG. 13 , when the capturing camera  30  is the fixed camera, the image determination unit  126  employs the technique of determining the region selected from the captured image as the display image. 
     In addition, as shown in  FIG. 13 , when the imaging device  30  is the direction adjustable camera, the image determination unit  126  determines the capturing direction of the imaging device  30  that provides the captured image, and determines the captured image captured in the determined capturing direction as the display image. For example, the image determination unit  126  determines the capturing direction of the imaging device  30  based on the composition decided by the composition decision unit  125 . In particular, the image determination unit  126  may determine the direction in which an image closest to the composition decided by the composition decision unit  125  can be captured as the capturing direction of the imaging device  30 . 
     In addition, as shown in  FIG. 13 , when the imaging device  30  is the angle-of-view adjustable camera, the image determination unit  126  determines the angle of view of the imaging device  30  that provides the captured image, and determines the captured image captured at the determined angle of view as the display image. For example, the image determination unit  126  determines the angle of view of the imaging device  30  based on the composition decided by the composition decision unit  125 . In particular, the image determination unit  126  may determine the angle of view at which an image closest to the composition decided by the composition decision unit  125  can be captured as the angle of view of the imaging device  30 . 
     In addition, as shown in  FIG. 13 , when captured images provided from the respective imaging devices  30  are acquired by the image acquisition unit  111 , the image determination unit  126  determines the captured image selected from the captured images provided from the respective imaging devices  30  as the display image. For example, the image determination unit  126  determines the captured image selected from the captured images provided from the respective imaging devices  30  as the display image based on the composition decided by the composition decision unit  125 . In particular, the image determination unit  126  may select the captured image closest to the composition decided by the composition decision unit  125  from the captured images provided from the respective imaging devices  30 , and may determine the selected captured image as the display image. 
     In addition, in the example described above, the image closest to the composition decided by the composition decision unit  125  is used as the display image. However, when only an image having a difference exceeding a threshold between the image and the composition is acquired, the display image may not be determined In this case, it is also assumed that it is difficult for the image determination unit  126  to determine the display image based on the composition decided by the composition decision unit  125 . Accordingly, in such a case, the image determination unit  126  may determine the display image based on detection data other than the composition. 
     The detection data is not particularly limited. For example, it is possible for the image determination unit  126  to determine an image in which a position closest to the user position specified by the specification unit  114  based on the sensor data is captured as the display image. In addition, the image determination unit  126  may determine an image having the smallest number of persons shown in the image as the display image, or may determine an image having the smallest density of persons shown in the image as the display image. Further, the image determination unit  126  may determine an image in which the largest range is captured as the display image, or may determine the captured image that is most frequently used as the display image. 
     In addition, when the imaging device  30  is not present or the like, it is also assumed that the captured image is not acquired by the image acquisition unit  111 . In such a case, the image determination unit  126  may determine an image according to information indicating the destination of the user and the user position detected based on data other than the captured image as the display image. 
     For example, when position information of the user is acquired based on the signal received by a sensor receiving GPS signals, the image determination unit  126  may use the position information and information indicating the destination of the user to search an image from a web page or the like. The image may be a plotted map, or may be a captured photograph. In addition, when plural images are acquired by searching, the image determination unit  126  may determine a new image generated by synthesizing a plurality of images as the display image. 
       FIG. 14  is a diagram illustrating an example of the determined display image. As shown in  FIG. 14 , the display control unit  160  may add an object Obj 1  indicating a distance from the user position Pu as a reference to the destination of the user, an object Obj 2  indicating an estimated time of arriving in the destination and a distance to the destination, and so forth, to the display image D 2 . 
     In addition, as shown in  FIG. 14 , the display control unit  160  may add an object Obj 3  indicating a bearing, an object Obj 4  indicating a direction of the destination of the user using the user position Pu as the reference, and so forth, to the display image D 2 . In addition, as shown in  FIG. 14 , the display control unit  160  may add an object Obj 5  indicating a direction of the destination of the user using the user position Pu as the reference to the user position Pu and may add an object Obj 6  indicating the user position Pu to a position of the user in the display image D 2 . 
     Next, a flow of the composition decision operation of the information processing device  10  will be described.  FIG. 15  is a flowchart illustrating the flow of the composition decision operation of the information processing device  10 . As shown in  FIG. 15 , when the captured image is not acquired by the image acquisition unit  111  (“No” in step S 21 ), the recognition unit  121  recognizes the user from data other than the captured image (step S 22 ), and the flow proceeds to step S 26 . 
     On the other hand, when the captured image is acquired by the image acquisition unit  111  (“Yes” in step S 21 ), the recognition unit  121  recognizes the user from the captured image (step S 23 ), and the condition decision unit  122  decides whether or not a condition for capturing is prepared (step S 24 ). When the condition decision unit  122  decides that the condition for capturing is prepared (“Yes” in step S 24 ), the flow proceeds to step S 26 . On the other hand, when the condition decision unit  122  decides that the condition for capturing is not prepared (“No” in step S 24 ), the information addition unit  123  adds information indicating that the condition is not prepared to the captured image (step S 25 ), and the flow proceeds to step S 26 . 
     When the flow proceeds to step S 26 , the mode setting unit  124  sets any one of the first mode and the second mode (step S 26 ), and the composition decision unit  125  decides the composition in accordance with the mode set by the mode setting unit  124  based on the recognition result of the recognition unit  121  (step S 27 ). The image determination unit  126  then tries to determine the display image based on the composition decided by the composition decision unit  125  (step S 28 ). 
     In this case, when the display image is determined by the image determination unit  126  (“Yes” in step S 29 ), the flow proceeds to step S 31 . On the other hand, when the display image is not determined by the image determination unit  126  (“No” in step S 29 ), the image determination unit  126  determines the display image based on detection data other than the composition (step S 30 ), and the flow proceeds to step S 31 . The display control unit  160  controls the portable terminal  20  such that the display image determined by the image determination unit  126  is displayed in the portable terminal  20  (step S 31 ), and the flow is finished. 
     As described above, according to the composition decision function of the information processing device  10 , the composition is decided based on information indicating the destination of the user and the recognition result of the user based on the captured image. In addition, the display image is determined based on the determined composition. In accordance with such control, the display image having higher convenience is expected to be determined for the user. 
     3. Depiction Change Function  
     Next, the depiction change function of the information processing device  10  will be described. As described above, it is assumed that the information processing device  10  provides a network service and the network service is provided to the portable terminal  20  when the user U logs into the network service via the portable terminal  20 . 
     Logging into the network service is not limited to the user U, and may be performed by another user. In other words, there is a possibility that a person shown in the captured image (hereinafter, also referred to as a subject) enjoy the network service. Hereinafter, a function of easily finding the change in state of the subject in the network service shown in the captured image will be described. In addition, the subject shown in the captured image may be recognized by the recognition unit  121  using the same technique as the technique specified by the user. 
     The change detection unit  131  detects the change in state of the subject in the network service recognized from the captured image acquired by the image acquisition unit  111 . The state in the network service is not particularly limited. For example, the state may be information indicating whether or not the portable terminal of the subject logs into the network service. In other words, when the change detection unit  131  detects that the portable terminal of the subject has logged into the network service, the depiction change unit  132  may change the depiction of the subject shown in the captured image. 
     In addition, the state in the network service may be information indicating whether or not a specific process is carried out while being logged into the network service. In other words, when the change detection unit  131  detects that the movement of the portable terminal of the subject in the network service satisfies a predetermined condition, the depiction change unit  132  may change the depiction of the subject shown in the captured image. The predetermined condition is not particularly limited. For example, the predetermined condition may be a condition that a message was written, and may be a condition that the message was written within a predetermined time. 
     When the change in state is detected by the change detection unit  131 , the depiction change unit  132  changes the depiction of the subject shown in the captured image. A region of the subject shown in the captured image may be figured out in any way by the depiction change unit  132 . For example, when a difference between the captured image in which the subject is not shown and the captured image in which the subject is shown occurs, the differential region therebetween may be found as the region of the subject. In addition, a method of changing the depiction of the subject is not particularly limited. 
     Further, when the user is specified by the specification unit  114 , the depiction change unit  132  may change the depiction of the user by emphasizing the user shown in the captured image. In addition, a method of emphasizing the user is not particularly limited. The display control unit  160  controls the portable terminal  20  such that the display image that may be obtained by causing the depiction change unit  132  to change the depiction is displayed in the portable terminal  20 . 
     Techniques of emphasizing the user will be described with reference to drawings.  FIG. 16  is a diagram illustrating an example of the depiction change function of the information processing device  10 . As shown in  FIG. 16 , the depiction change unit  132  may emphasize the user U by decreasing the definition of subjects other than the user U in the captured image. 
       FIG. 17  is a diagram illustrating another example of the depiction change function of the information processing device  10 . As shown in  FIG. 17 , the depiction change unit  132  may emphasize the user U by emphasizing an outline L 1  of the user U in the captured image. For example, the user U may be emphasized by making the outline L 1  of the user U thick in the captured image. 
       FIG. 18  is a diagram illustrating another example of the depiction change function of the information processing device  10 . As shown in  FIG. 18 , the depiction change unit  132  may emphasize the user U by changing the pixel value of subjects other than the user U in the captured image. For example, the user U may be emphasized by decreasing the brightness of subjects other than the user U in the captured image. 
       FIG. 19  is a diagram illustrating another example of the depiction change function of the information processing device  10 . As shown in  FIG. 19 , the depiction change unit  132  may emphasize the user U such that the user U is zoomed in on and displayed in the captured image. In addition, as shown in  FIG. 19 , the depiction change unit  132  may zoom out and display the user U after the zoom-in display is carried out. 
       FIG. 20  is a diagram illustrating another example of the depiction change function of the information processing device  10 . The depiction change unit  132  may emphasize the user by displaying an object at a position of the user or a position near the user in the captured image. The range of surrounding the user may be defined in advance. In the example shown in  FIG. 20 , the depiction change unit  132  emphasizes the user U by adding objects Obj 8  and Obj 9  to the position around the user in the captured image. 
     In addition, the degree of emphasizing the user U may be constant, or may be changed depending on the situation. For example, the depiction change unit  132  may change the degree of emphasizing the user in accordance with the size of the user in the captured image.  FIG. 21  is a diagram illustrating another example of the depiction change function of the information processing device  10 . For example, the depiction change unit  132  may increase the degree of emphasizing the user to the same extent to which the size of the user in the captured image decreases. 
     In the example shown in  FIG. 21 , as the position of the user U is farther from the imaging device  30 , the size of the user U in the captured image is smaller. Therefore, the depiction change unit  132  increases the degree of emphasizing the user U having a small size by making the outline LO of the user U having the small size thicker than the outline L 1  of the user U having a large size. In this way, it is easier to find the user in the captured image by increasing the degree of emphasizing the user to the same extent to which the size of the user in the captured image decreases. 
     In addition, the user U may be hidden behind other objects in the captured image. For example, when a portion of the user or the entire user is hidden and shown by other objects in the captured image, the depiction change unit  132  may change the technique of emphasizing the user.  FIG. 22  is a diagram illustrating another example of the depiction change function of the information processing device  10 . As shown in  FIG. 22 , a portion of the user U is hidden by other subjects in the captured image. The depiction change unit  132  thus changes the technique of emphasizing the user U by adding the object Obj 10  to a position around the user U in the captured image. The range of surrounding the user U may be defined in advance. 
     Next, a flow of depiction change operations of the information processing device  10  will be described.  FIG. 23  is a flowchart illustrating the flow of the depiction change operations of the information processing device  10 . As shown in  FIG. 23 , when the captured image is not acquired by the image acquisition unit  111  (“No” in step S 41 ), the flow is finished. On the other hand, when the captured image is acquired by the image acquisition unit  111  (“Yes” in step S 41 ), the subject is recognized from the captured image by the recognition unit  121  (step S 42 ). The change detection unit  131  then decides whether or not the change in state of the subject in the network service is detected (step S 43 ). 
     When the change in state of the subject in the network service is not detected by the change detection unit  131  (“No” in step S 43 ), the flow is finished. On the other hand, when the change in state of the subject in the network service is detected by the change detection unit  131  (“Yes” in step S 43 ), the depiction change unit  132  changes the depiction of the subject (step S 44 ). The display control unit  160  controls display of the display image that may be obtained by the depiction change unit  132  that changes the depiction (step S 45 ), and the flow is finished. 
     As described above, according to the depiction change function of the information processing device  10 , the change in state of the subject in the network service recognized from the captured image is detected. In addition, when the change in state is detected, the depiction of the subject shown in the captured image is changed. According to such control, the change in state of the subject in the network service shown in the captured image is expected to be easily found. 
     4. Conclusion 
     As described above, according to the user specification function of the information processing device  10  of the present embodiment, the information processing device  10  including the image acquisition unit  111  that acquires the captured image, the parameter acquisition unit  112  that acquires the parameter associated with the user of the portable terminal  20 , the specification unit  114  that specifies the user from the captured image based on the parameter, and the display control unit  160  that control the portable terminal such that information indicating the movement of the user necessary to specify the user is displayed in the portable terminal  20  when the user is not specified by the specification unit  114  is provided. 
     According to the configuration described above, when the user is not specified by the specification unit  114 , the information indicating the movement of the user necessary to specify the user is displayed by the portable terminal  20 , and the user who has viewed the information can move so as to specify the user from the captured image. Accordingly, the user is expected to be easily specified from the captured image. 
     In addition, according to the composition decision function of the information processing device  10  of the present embodiment, the information processing device  10  including the image acquisition unit  111  that acquires the captured image, the composition decision unit  125  that decides the composition based on information indicating the destination of the user and the recognition result of the user based on the captured image, and the image determination unit  126  that determines the display image based on the composition decided by the composition decision unit  125  is provided. 
     According to such a configuration, the composition is decided based on the information indicating the destination of the user and the recognition result of the user based on the captured image, and the display image is determined based on the composition. For example, the composition is decided based on information indicating the user position and information indicating the destination by the composition decision unit  125 . Accordingly, the display image having higher convenience is expected to be determined for the user. 
     In addition, according to the depiction change function of the information processing device  10  of the present embodiment, the information processing device  10  including the image acquisition unit  111  that acquires the captured image, the change detection unit  131  that detects the change in state of the subject in the network service recognized from the captured image, and the depiction change unit  132  that changes the depiction of the subject shown in the captured image when the change in state is detected by the change detection unit  131  is provided. 
     According to such a configuration, when the change in state of the subject in the network service is detected, the depiction of the subject shown in the captured image is changed. It is possible for the user who has viewed the captured image to easily find the change in state of the subject in the network service from the captured image. 
     It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and alterations may occur depending on design requirements and other factors insofar as they are within the scope of the appended claims or the equivalents thereof. 
     In addition, for example, an example in which the information processing device  10  includes all of the processing control unit  100  and the display control unit  160  has been described. However, a device other than the information processing device  10  may have a portion of the blocks or the entire blocks. For example, a server may have a portion of the processing control unit  100  and the display control unit  160  and the portable terminal  20  may have the remaining portion. In this way, the technique of the present disclosure can also be applied to cloud computing. 
     In addition, it is not necessary for each step in operations of the information processing device  10  of the present specification to be processed in a time series in the order described in the flowchart. For example, each step in the operations of the information processing device  10  may be processed in an order different from the order described in the flowchart, or may be processed in parallel. 
     In addition, it is possible to create a computer program causing hardware such as a CPU, a ROM, and a RAM embedded in the information processing device  10  to exert the same function as the configuration of the information processing device  10  described above. In addition, a storage medium in which the computer program is stored is also provided. 
     Additionally, the present technology may also be configured as below. 
     (1) 
     An information processing device including: 
     an image acquisition unit configured to acquire a captured image; 
     a parameter acquisition unit configured to acquire a parameter associated with a user of a portable terminal; 
     a specification unit configured to specify the user from the captured image on the basis of the parameter; and 
     a display control unit configured to perform control in a manner that, in a case where the user is not specified by the specification unit, information indicating a movement of the user necessary to specify the user is displayed in the portable terminal. 
     (2) 
     The information processing device according to (1), 
     wherein the parameter acquisition unit acquires a face image of the user as the parameter, and 
     wherein the specification unit specifies, as the user, a subject extracted on the basis of checking of a facial region shown in the captured image against the face image of the user. 
     (3) 
     The information processing device according to (2), 
     wherein, in a case where the user is not specified by the specification unit, the display control unit performs control in a manner that information prompting the user to turn a face toward an imaging device imaging the captured image is displayed in the portable terminal as the information indicating a movement of the user necessary to specify the user. 
     (4) 
     The information processing device according to (1), 
     wherein the parameter acquisition unit acquires sensor data representing a movement of the user as the parameter, and 
     wherein the specification unit specifies, as the user, a subject extracted on the basis of checking of a movement of a subject detected from the captured image against the sensor data. 
     (5) 
     The information processing device according to (4), 
     wherein, in a case where the user is not specified by the specification unit, the display control unit performs control in a manner that information prompting the user to move the portable terminal is displayed in the portable terminal as the information indicating a movement of the user necessary to specify the user. 
     (6) 
     The information processing device according to any one of (1) to (5), further including 
     a candidate extraction unit configured to extract a candidate for the user from the captured image, 
     wherein the specification unit specifies the user on the basis of the parameter from the candidate for the user extracted by the candidate extraction unit. 
     (7) 
     The information processing device according to (6), 
     wherein the parameter acquisition unit acquires coordinates on the captured image designated by an operation from the user as the parameter, and 
     wherein the candidate extraction unit extracts a subject selected on the basis of the coordinates as the candidate for the user. 
     (8) 
     The information processing device according to (7), 
     wherein, in a case where the user is not specified by the specification unit, the display control unit performs control in a manner that the information indicating a movement of the user necessary to specify the user is displayed in the portable terminal on the basis of a position of the candidate for the user extracted by the candidate extraction unit and a position of the imaging device that is set in advance. 
     (9) 
     The information processing device according to any one of (1) to (8), 
     wherein, in a case where the user is not specified by the specification unit, the display control unit performs control in a manner that information indicating that the user is specified by another specification technique is displayed in the portable terminal, and
 
wherein the specification unit specifies a subject extracted by the other specification technique as the user.
 
(10)
 
     The information processing device according to any one of (1) to (9), further including 
     an authentication unit configured to perform an authentication process on the portable terminal specified by the specification unit. 
     (11) 
     An information processing method including: 
     acquiring a captured image; 
     acquiring a parameter associated with a user of a portable terminal; 
     specifying the user from one or more subjects shown in the captured image on the basis of the parameter; and 
     performing control in a manner that, in a case where the user is not specified, information indicating a movement of the user necessary to specify the user is displayed in the portable terminal. 
     (12) 
     A program for causing a computer to function as an information processing device including: 
     an image acquisition unit configured to acquire a captured image; 
     a parameter acquisition unit configured to acquire a parameter associated with a user of a portable terminal; 
     a specification unit configured to specify the user from one or more subjects shown in the captured image on the basis of the parameter; and 
     a display control unit configured to perform control in a manner that, in a case where the user is not specified by the specification unit, information indicating a movement of the user necessary to specify the user is displayed in the portable terminal. 
     REFERENCE SIGNS LIST 
     
         
           1  information processing system 
           10  information processing device 
           20  portable terminal 
           30  imaging device 
           40  network 
           50  storage unit 
           100  processing control unit 
           111  image acquisition unit 
           112  parameter acquisition unit 
           113  candidate extraction unit 
           114  specification unit 
           115  authentication unit 
           121  recognition unit 
           122  condition decision unit 
           123  information addition unit 
           124  mode setting unit 
           125  composition decision unit 
           126  image determination unit 
           131  change detection unit 
           132  depiction change unit 
           160  display control unit