Patent Document

TECHNICAL FIELD 
       [0001]    The present invention relates to a captured-image recognition apparatus, a captured-image recognition system, and a captured-image recognition method, and for example, to an image recognition technique performed by a surveillance camera system and an in-vehicle camera system. 
       BACKGROUND ART 
       [0002]    In recent years, as a technique supporting human visual observation, image recognition techniques using circuits are used in many fields. For example, a surveillance camera system uses an image recognition technique to perform congestion-degree detection, detection of an intruder, or recognition of an intruder&#39;s face. In addition, an in-vehicle camera performs traffic lane detection, traffic sign detection, or pedestrian detection, for example. 
         [0003]    Camera systems in recent years have many functions in association with diversification of the uses. Camera systems are also proposed which select, from the many functions, a function suitable for the state of a captured image. 
         [0004]    For example, a multi-camera system disclosed by Patent Literature 1 selects, from multiple functions, the optimal function suitable for the state of images captured by multiple cameras. 
         [0005]    Specifically, when the visual fields of two cameras are overlapped, at least one function is performed among a stereo distance measurement function, a tridimensional-shape measurement function, a panoramic image generation function, a blind-spot-imaging function, and a person action analysis function. On the other hand, when the visual fields of two cameras are not overlapped, objects for detection are detected using respective captured images of the two cameras. 
         [0006]    As described above, some conventional camera systems can select and perform a function suitable for the state of a captured image from multiple functions included in the systems to thereby perform the most suitable process for the captured image. 
       CITATION LIST 
     Patent Literature 
       [0007]    PTL 1 Japanese Patent Application Laid-Open No. 2009-77092 
       SUMMARY OF INVENTION 
     Technical Problem 
       [0008]    As described above, in camera systems including multiple functions, it is very important to select and perform a function suitable for the state of a captured image from multiple functions in order to obtain a highly precise execution result. 
         [0009]    However, about image recognition performed in surveillance camera systems or in-vehicle camera systems, it have not been sufficiently examined which recognition function can be used among multiple image recognition functions to fully utilize the recognition functions included in the systems and obtain a highly precise recognition result. 
         [0010]    Especially, the technical development of image recognition has been remarkable, and a large number of image recognition techniques have been developed in recent years. Therefore, it is important to select the optimal image recognition function from among the techniques. However, special knowledge is required in order to manually select the optimal image recognition function. In other words, it is difficult for ordinary camera users to fully elicit the performance of recognition functions. Moreover, a criterion for determining which recognition function is selected for what kind of image has not been fully examined yet. Thus, even if special knowledge is provided, the performance of recognition functions cannot necessarily be elicited fully. 
         [0011]    The present invention is made in consideration of the abovementioned points. It is an object of the present invention to provide a captured-image recognition apparatus, a captured-image recognition system, and a captured-image recognition method that can fully elicit the performance of image recognition functions. 
       Solution to Problem 
       [0012]    A captured-image recognition apparatus according to an aspect of the present invention includes: an imaging state estimation section that estimates a visual field ratio of a target size in a captured image to a field-angle size of the captured image; a candidate application selection section that selects at least one candidate of an image recognition application based on the estimated visual field ratio; and an image recognition processing section that executes an image recognition application selected from among the candidate applications. 
         [0013]    A captured-image recognition system according to an aspect of the present invention includes: a server apparatus that includes a database storing a plurality of image recognition application programs therein; and a network camera to be connected to the server apparatus in a network, in which: the server apparatus selects a candidate of an image recognition application from among the plurality of stored image recognition application programs, based on a visual field ratio of a target size in an image captured by the network camera to a field-angle size of the captured image; the server apparatus presents information on the selected candidate of the image recognition application to a user of the network camera; and the server apparatus transmits a program for the image recognition application selected by the user to the network camera. 
         [0014]    A captured-image recognition method according to an aspect of the present invention includes: an imaging state estimation step of estimating a visual field ratio of a target size in a captured image to a field-angle size of the captured image; a candidate application selection step of selecting at least one candidate of an image recognition application based on the estimated visual field ratio; and an image recognition processing step of executing an image recognition application selected from among the candidate applications. 
       Advantageous Effects of Invention 
       [0015]    The present invention can provide a captured-image recognition apparatus, a captured-image recognition system, and a captured-image recognition method that can fully elicit the performance of image recognition functions. 
     
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         [0016]      FIG. 1  illustrates the relationship between a distance from a camera to an object for imaging and a visual field ratio; 
           [0017]      FIGS. 2A and 2B  illustrate the configuration of the present embodiment described in association with the visual field ratio of the camera and captured images,  FIG. 2A  illustrates the relationship between the captured images and the visual field ratio of the camera, and  FIG. 2B  is a block diagram illustrating the configuration according to the present embodiment; 
           [0018]      FIG. 3  illustrates the relationship among the visual field angle of the camera, a distance from the camera to the object (target) for imaging, and the visual field ratio; 
           [0019]      FIG. 4  is a flow chart for explaining operations of a captured-image recognition system; 
           [0020]      FIGS. 5A and 5B  illustrate an example display of availability of equipment resources,  FIG. 5A  illustrates availability of equipment resources when one image recognition application is selected, and  FIG. 5B  illustrates availability of equipment resources when two image recognition applications are selected; 
           [0021]      FIG. 6  is a flow chart for illustrating a procedure for performing the display of availability of an equipment resource; 
           [0022]      FIG. 7  is a block diagram for illustrating an example network configuration according to the present embodiment; 
           [0023]      FIG. 8  is a block diagram for illustrating another example network configuration according to the present embodiment; and 
           [0024]      FIG. 9A  illustrates an example process where a surveillance camera is mounted at a position near a monitored region, and  FIG. 9B  illustrates an example process where a surveillance camera is mounted at a position distant from a monitored region. 
       
    
    
     DESCRIPTION OF EMBODIMENTS 
       [0025]    The inventors of the present invention have focused attention on the fact that some image recognition functions are suitable for a captured image while others are not suitable for it, depending on the captured image. The inventors have minutely examined what kinds of image recognizing processes are suitable for what kinds of captured images. In addition, the inventors have minutely examined indices for selecting image recognition functions to be applied to captured images. 
         [0026]    As a result of the examination, the inventors have found that using a visual field ratio of a target size in the captured image to the field-angle size of the captured image as an index for selecting an image recognizing function makes it possible to perform an image recognizing process suitable for a captured image. 
         [0027]    An aspect of the present invention includes; estimating the ratio of a target size in a captured image to the field-angle size of the captured image; selecting at least one candidate of an image recognition application based on the estimated ratio; and executing an image recognition application selected from among the candidate applications. 
         [0028]    A method of detecting a ratio of a target size in a captured image to the field-angle size of the captured image is not limited especially. However, the following embodiment describes a method using the visual field ratio of a camera as the ratio. This is based on focusing attention on the general phenomenon of decreasing a ratio of a target size in a captured image to the field-angle size of the captured image with an increase in the visual field angle of a camera, and decreasing the ratio of the size of the target in the captured image to the field-angle size of the captured image with an increase in a distance from the camera to an object for imaging, as illustrated in  FIG. 1 . 
         [0029]    The following embodiment explains an exemplary case where a target for recognition is a person. However, the target may not be a person and may be, for example, a vehicle or other objects. 
         [0030]    The embodiment of the present invention will be described in detail with reference to the accompanying drawings. 
         [0031]    [1] Configuration 
         [0032]      FIGS. 2A and 2B  illustrate the configuration of the present embodiment described in association with the visual field ratio of the camera and captured images P 1  to P 5 .  FIG. 2A  illustrates the relationship between captured images P 1  to P 5  and the visual field ratio of the camera.  FIG. 2B  is a block diagram illustrating the configuration according to the present embodiment. 
         [0033]    As is apparent from  FIG. 2A , the target size (the size of the person appearing in the example of  FIG. 2A ) in the captured image to the image size of the captured image increases with an increase in the visual field ratio. 
         [0034]      FIG. 3  illustrates the relationship among the visual field angle of the camera, a distance from the camera to the object (target) for imaging, and the visual field ratio. As is apparent from the drawing, the visual field ratio for an equal visual field angle increases with a decrease in a distance from the camera. 
         [0035]    As illustrated in  FIG. 2B , captured image recognition system  10  of the present embodiment includes camera section  20 , process selecting section  30 , image recognition processing section  40 , user terminal  50 , and zoom control section  60 . 
         [0036]    Image recognition processing section  40  is provided in a server apparatus. Alternatively, image recognition processing section  40  may be provided in camera section  20 . Process selecting section  30  is provided in the server apparatus. Alternatively, process selecting section  30  may be provided in camera section  20  or user terminal  50 . 
         [0037]    Image recognition processing section  40  holds multiple application programs (hereinafter also referred to as application simply) for implementing multiple image recognition functions, and executes an image recognition application selected by a user from among the multiple image recognition applications. 
         [0038]    In the present embodiment, image recognition processing section  40  stores applications performing congestion detection, person detection, movement recognition, facial recognition, visual-line estimation, movement-line detection, cloth search, sex/age estimation, and facial-expression estimation as the image recognition applications, and executes the respective applications in congestion detecting section  41 , person detecting section  42 , movement-recognition section  43 , facial-recognition section  44 , visual-line estimation section  45 , movement-line detecting section  46 , cloth search section  47 , sex/age estimation section  48 , and facial-expression estimation section  49 . 
         [0039]    Each application includes a set visual field ratio suitable for the operation of the application. The set visual field ratio is beforehand determined based on, for example, experiments or the property of application. Alternatively, a suitable visual field ratio varies depending on the resolution of the camera and may therefore be changed appropriately depending on resolution information on the camera. For example, since the face of the person can be electronically expanded if the resolution is high even in the case of a low visual field ratio as illustrated in P 2  of  FIG. 2A , the proper range of facial-recognition section  44  may be extended in the direction in which the visual field ratio decreases. 
         [0040]    Process selecting section  30  selects a candidate of an image recognition application which is properly executable for the current captured image from among multiple image recognition applications which are executable in image recognition processing section  40 , and presents these candidate applications to the user. 
         [0041]    Process selecting section  30  receives captured image S 1  from camera section  20  at visual-field-ratio estimation section  31 . Visual-field-ratio estimation section  31  estimates the visual field ratio of camera section  20  using captured image S 1 . In present embodiment, visual-field-ratio estimation section  31  performs a person detecting process and a face detecting process, and estimates the visual field ratio based on the detected person size and face size. For example, the visual field ratio can be estimated based on the person size and face size relative to field-angle size of captured image S 1 . Specifically, the visual field ratio may be estimated to be smaller for a lower ratio of the person size or face size to the field-angle size. 
         [0042]    However, how a visual field ratio is estimated is not restricted to this method. For example, a person detecting process and a face detecting process may be performed to estimate a visual field ratio based on the degree of success or failure of each detecting process. For example, a visual field ratio α may be estimated to be small for a higher success rate of the person detection and to be large for a higher success rate of the face detection. 
         [0043]    Moreover, person detecting processes using multiple templates having different sizes may be performed to estimate a visual field ratio based on the degrees of success or failure of the detecting processes using the respective sizes. For example, by using three templates having respective large, medium, and small sizes, a visual field ratio may be estimated to be larger for a higher success rate of the person detection with the large template and to be smaller for a higher success rate of the person detection with the small template. 
         [0044]    For example, visual-field-ratio estimation section  31  may also estimate a visual field ratio based on a parameter from camera section  20 . Specifically, the visual field ratio can be found from information on the visual field angle of the camera and information on a positional relationship (distance) between the camera and the target, by using the relationship illustrated in  FIG. 1 . 
         [0045]    Estimated visual field ratio α is sent to candidate application selection section  32 . Application selection section  32  selects a candidate of an image recognition application based on visual field ratio α. A candidate of an image recognition application is defined as an image recognition application which is properly executable for the current captured image among multiple image recognition applications which are executable in image recognition processing section  40 . 
         [0046]      FIG. 2B  illustrates which image recognition application is suitable for which visual field ratio. The exemplary case of the drawing illustrates that image recognition application processes suitable for visual field ratio α1 are congestion detection, person detection, movement recognition, and movement-line detection. Therefore, candidate application selection section  32  selects, for visual field ratio α1, congestion detection, person detection, movement recognition, and movement-line detection as the candidates of image recognition applications. 
         [0047]    In the above description, the zoom magnification of the camera is fixed. However, when the zoom magnification is variable, the visual field ratio also varies as the visual field angle of the camera is changed.  FIG. 2B  illustrates visual field ratio α1 indicated by a straight line. However, when the zoom magnification of the camera is variable, α1 may be indicated by a straight line having a width in the right and left direction, the width corresponding to the zoom range. 
         [0048]    The information on the candidate application selected by candidate application selection section  32  of process selecting section  30  is sent to display section  51  of user terminal  50  as proper process recommendation information S 2 . Thereby, the candidates of the image recognition applications which are properly executable in image recognition processing section  40  are displayed for the current captured image on display section  51 , and the user can select, from among the candidates, an image recognition application which the user intends to execute. 
         [0049]    In addition to selecting candidate applications, candidate application selection section  32  may display the rankings of the selected candidate applications. That is, as illustrated in  FIG. 2B , each image recognition application includes an effective visual field ratio in a visual field region which allows an intended application to be executed, and a proper visual field ratio in a visual field region which allows the application to be executed more properly in the effective visual field ratio. The proper visual field ratio is included in the range of a part of the effective visual field ratio. 
         [0050]    Candidate application selection section  32  may judge whether the selected candidate application is a merely executable application or a properly executable application, and cause display section  51  to display the result. For example, a merely executable application may be displayed in a “utilizable application list,” and an application which can be more properly performed may be displayed in a “proper application list.” 
         [0051]    In practice, the user selects an image recognition application that he or she wishes to execute, while browsing image recognition applications displayed on display section  51 , and inputs the selection result through input section  52 . The information on the selected image recognition application is sent out to image recognition processing section  40 . Image recognition processing section  40  then executes the image recognition application selected by the user. 
         [0052]    The information on the image recognition application selected by the user is also sent to zoom control section  60 . Zoom control section  60  controls the zooming (focal length) so that the visual field ratio of camera section  20  can be optimal for the selected image recognition application. For example, when congestion detection is selected by the user as an image recognition application under visual field ratio α1 as illustrated in  FIG. 2B , a visual field ratio smaller than α1 is more preferred for congestion detection. Therefore, zoom control section  60  controls the zooming so that the visual field ratio (visual field angle) can be smaller than its current value. 
         [0053]    [2] Operations of Captured-Image Recognition System 
         [0054]    Next, operations of captured-image recognition system  10  will be described with reference to  FIG. 4 . 
         [0055]    First, initial setting for camera section  20  is performed at step ST 11 . At step ST 12 , camera section  20  captures an image. At step ST 13 , visual-field-ratio estimation section  31  estimates visual field ratio α. In step ST 14 , the estimated visual field ratio α is stored. Estimated visual field ratio α may be stored, for example, in a memory provided in visual-field-ratio estimation section  31 . 
         [0056]    Meanwhile, captured-image recognition system  10  starts an image recognition selection process at step ST 21 . First, candidate application selection section  32  acquires a list of image recognition applications from image recognition processing section  40  at step ST 22 . The list of image recognition applications is a list of the image recognition applications executable in image recognition processing section  40 . In the example of  FIG. 2B , the image recognition application list includes congestion detection, person detection, movement recognition, facial recognition, visual-line estimation, movement-line detection, cloth search, the sex/age estimation, and facial-expression estimation. 
         [0057]    At step ST 23 , candidate application selection section  32  selects at least one or more image recognition applications suitable for current visual field ratio α from the image recognition application list, and causes display section  51  to display the selected image recognition applications as a “candidate application.” In the example of  FIG. 2B , since an image is captured with visual field ratio α1, display section  51  displays that congestion detection, person detection, movement recognition, and movement-line detection are executable. 
         [0058]    At step ST 24 , an image recognition application is selected by the user using input section  52 . 
         [0059]    At step ST 25 , in order for image recognition processing section  40  to execute an image recognition application selected by the user, an image recognition application program is acquired from the outside of the system through a network, or a module is changed to the selected image recognition application program when the programs are held beforehand. 
         [0060]    At step ST 26 , zoom control section  60  controls the zoom of camera section  20  so as to set a visual field ratio optimal for the image recognition application selected by the user. 
         [0061]    At step ST 27 , camera section  20  captures an image. 
         [0062]    At step ST 28 , image recognition processing section  40  performs an image recognition process with the image recognition application selected by the user. 
         [0063]    [3] Display of Equipment Resource Availability 
         [0064]    In this embodiment, it is preferable to cause display section  51  to display availability of an equipment resource in addition to candidate applications because the user can use the information as an index for selecting an image recognition application. 
         [0065]      FIGS. 5A and 5B  illustrate an example display of availability of equipment resources.  FIG. 5A  illustrates availability of equipment resources when one image recognition application is selected (for example, when person detection is selected).  FIG. 5B  illustrates availability of equipment resources when two image recognition applications are selected (for example, when person detection and movement-line detection are selected). The user can view availability of equipment resources to thereby find the remaining quantity of an equipment resource. This can be used as an index for selecting an image recognition application. 
         [0066]      FIG. 6  illustrates a procedure for performing the display of availability of an equipment resource. In  FIG. 6 , the same processes as those in  FIG. 4  are assigned the same reference signs as those in  FIG. 4 . The procedure in  FIG. 6  is configured by adding steps ST 30 , ST 31 , and ST 32  to the procedure in  FIG. 4 . 
         [0067]    At step ST 30 , the availability of equipment resources is displayed as illustrated in  FIG. 5 . Step ST 31  judges a resource required to execute the image recognition application selected at Step ST 24 . This judgment may be made by, for example, candidate application selection section  32  sending a query to image recognition processing section  40  (for example, a server). In step ST 32 , whether or not selection of application is completed is judged. If the selection is not completed (step ST 32 ; No), the process returns to step ST 30 . In step ST 30 , the availability of an equipment resource obtained by adding the required resource judged at Step ST 31  is displayed in addition to the resource processed at previous step ST 30 . That is, if the contents displayed at previous step ST 31  are something like  FIG. 5A , the contents displayed in step ST 31  will be something like  FIG. 5B . 
         [0068]    [4] Network Configuration 
         [0069]    Next, a favorable network configuration of captured-image recognition system  10  will be described with reference to  FIG. 7  and  FIG. 8 . Here, elements corresponding to those in  FIG. 2B  are assigned the same reference signs in  FIG. 7  and  FIG. 8 . 
         [0070]    The network configuration in  FIG. 7  is an example configuration in which a network camera downloads and uses an image recognition application program through a network. 
         [0071]    The network configuration in  FIG. 7  includes server  100 , network camera (NW camera)  200  connected to server  100  in a network, and user terminal  50 . User terminal  50  may be provided in network camera  200 , may be provided in server  100 , or may be provided independently of network camera  200  and server  100 . However, user terminal  50  need be linked with network camera  200  in order to be provided independently. 
         [0072]    Server  100  includes candidate application selection section  32 , application database (application DB)  101  storing image recognition application programs, proper process recommendation query section  102 , receiving section  103 , and transmitting section  104 . 
         [0073]    Network camera  200  includes camera section  20 , visual-field-ratio estimation section  31 , image recognition processing section  40 , transmitting section  201 , receiving section  202 , and download application accumulation section (DL application accumulation section)  203 . 
         [0074]    After camera section  20  obtains captured image S 1 , visual-field-ratio estimation section  31   a  estimates and then transmits visual field ratio α to server  100  through transmitting section  201 . 
         [0075]    Server  100  receives visual field ratio α through receiving section  103 . Candidate application selection section  32  selects candidates of image recognition applications based on visual field ratio α and sends the selected candidates of image recognition applications to proper process recommendation query section  102 . Processing recommendation query section  102  causes display section  51  of user terminal  50  to display the selected candidates of image recognition applications. Moreover, proper process recommendation query section  102  receives information on an image recognition application selected by the user, from input section  52  of user terminal  50  and transmits the image recognition application program selected by the user to network camera  200  through transmitting section  104 . 
         [0076]    Network camera  200  receives the image recognition application program by receiving section  202  and accumulates the received program in download application accumulation section  203 . Image recognition processing section  40  performs an image recognition process using the image recognition application program downloaded in download application accumulation section  203 . 
         [0077]    In the example network configuration of  FIG. 8 , the network camera captures an image, and the server performs almost all of the other processes. 
         [0078]    In the configuration of  FIG. 8 , network camera  400  transmits captured image S 1  obtained by camera section  20  to server  300  through transmitting section  401 . 
         [0079]    Server  300  receives captured image S 1  at receiving section  303 , and inputs the received image to visual-field-ratio estimation section  31  and image recognition processing section  40 . Visual-field-ratio estimation section  31  estimates visual field ratio α, and candidate application selection section  32  selects candidates of image recognition applications suitable for visual field ratio α. Processing recommendation query section  302  causes display section  51  of user terminal  50  to display the selected candidates of image recognition applications. Moreover, proper process recommendation query section  302  receives information on an image recognition application selected by the user, from input section  52  of user terminal  50 , and sends this information to image recognition processing section  40 . Image recognition processing section  40  reads the image recognition application program selected by the user from application database (application DB)  301 , and performs an image recognition process on captured image S 1  using this image recognition application program. 
         [0080]    [5] Example Applications 
         [0081]    Example applications (business models) implementable by applying the configuration of the present invention will be described. 
         [0082]    When multiple image recognition application programs are sold as packaging software, users have to single-handedly select an image recognition application program considered to be suitable for, for example; how a camera is mounted, from multiple image recognition application programs, and has to perform performance evaluation of image recognition. If a user selects a wrong image recognition application program at this time, the performance of the image recognition cannot be evaluated in the optimal combination of the mounted camera and the prepared image recognition application program. 
         [0083]    In contrast to this, by using the configuration of the present embodiment, the users of a camera just need to purchase packaging software including multiple image recognition application programs, and an image recognition application program suitable for the camera is selected by visual-field-ratio estimation section  31  and candidate application selection section  32  without the need for users to select the image recognition application program suitable for the camera. Consequently, the performance evaluation using the image recognition application program suitable for the current camera state can be performed. As a result, for example, a user planning the purchase of a camera and an image recognition application program can consider the performance evaluation suitable for the camera state as the index of the purchase. 
         [0084]      FIGS. 9A and 9B  illustrate examples in which the processes of the present embodiment are applied to packaging software including image recognition applications for congestion-degree detection, movement-line detection, and the sex/age estimation. 
         [0085]      FIG. 9A  illustrates a case where a surveillance camera is mounted at a position near a monitored region (imaged region). At this time, based on a visual field ratio, image recognition applications for movement-line detection and the sex/age estimation are selected and executed. 
         [0086]      FIG. 9B  illustrates a case where a surveillance camera is mounted at a position distant from a monitored region (imaged region). At this time, based on a visual field ratio, image recognition applications for congestion-degree detection and movement-line detection are selected and executed. 
         [0087]    [6] Advantageous Effects 
         [0088]    As described above, the present embodiment includes visual-field-ratio estimation section  31 , candidate application selection section  32  configured to select candidates of image recognition applications based on an estimated visual field ratio, and image recognition processing section  40  configured to execute an image recognition application selected by a user from the candidate applications. Thereby, the image recognition application suitable for the current captured image can be presented to a user and causes the user to execute a proper image recognition application. Accordingly, the performance of the image recognition function can be sufficiently elicited. 
         [0089]    In addition, process selecting section  30  in the above-described embodiment can be constituted by a computer such as a personal computer including a memory and CPU. The function of each component constituting process selecting section  30  can be implemented by the CPU reading and executing a computer program stored on the memory. 
         [0090]    The disclosure of Japanese Patent Application No. 2011-006832, filed on Jan. 17, 2011, including the specification, drawings and abstract, is incorporated herein by reference in its entirety. 
       INDUSTRIAL APPLICABILITY 
       [0091]    The present invention is suitable for a case where an image recognition process is applied to an image captured by a camera such as a surveillance camera and an in-vehicle camera. 
       REFERENCE SIGNS LIST 
       [0000]    
       
           10  Captured-image recognition system 
           20  Camera section 
           30  Process selecting section 
           31  Visual-field-ratio estimation section 
           32  Candidate application selection section 
           40  Image recognition processing section 
           41  Congestion detecting section 
           42  Person detecting section 
           43  Movement-recognition section 
           44  Facial-recognition section 
           45  Visual-line estimation section 
           46  Movement-line detecting section 
           47  Cloth search section 
           48  Sex/age estimation section 
           49  Facial-expression estimation section 
           50  User terminal 
           51  Display section 
           52  Input section 
           60  Zoom control section 
           100 ,  300  Server 
           200 ,  400  Network camera 
         S 1  Captured image 
         S 2  Proper process recommendation information 
         α, α1 Visual field ratio

Technology Category: 5