Patent Description:
A technology for detecting an object included in an input image has been conventionally developed. Object detection is an important technology for realizing a function as an eye of a robot. Research regarding the object detection has been performed from object detection in which a detection target is specified, such as detection of a face or a person from an image. In recent years, a technology for detecting an object by using machine learning and deep learning has also been proposed (refer to, for example, Patent Literatures <NUM> to <NUM>). <CIT> discloses a system and method for an image-based object detection and classification. A convolutional neural network is applied to the image to obtain localization data to detect an object depicted in the image and to obtain classification data to classify the object. The image is annotated based on the classification and localization data. <CIT> discloses a sales registration apparatus including an image sensor. When the presence of a subject is detected within the image-capturing range of the image sensor, a frame image is created to acquire the identifier of a corresponding product. A feature quantity of the frame image is stored in a storage device in connection with the acquired identifier, and product information associated with the identifier is acquired from a product information DB to perform sales processing.

In a case where machine learning is used, by allowing an object position region detection unit that detects a position region of an object included in an image and a class identification unit that identifies a class (type) of the above-described object to learn in advance by using a variety of data (in multiple classes), it becomes possible to detect objects in multiple classes at the time of inference. However, there is a case where directly outputting a detection result of the objects in multiple classes is not effective depending on applications.

For example, in a traffic census based on an image of a road, in a case where a purpose is only to count each of "person", "vehicle", and "animal" as objects passing through the road, it is necessary to detect a large class of "person", "vehicle", and "animal", and at that time, it is not necessary to distinguish "car", "bus", and "truck", which are small classes of "vehicle", for example. Therefore, in such applications, it is not effective to output a detection result regarding objects in fine multiple classes (information such as "car" and "bus" in the above-described example).

In contrast, by allowing the above-described object position region detection unit and class identification unit to learn in advance by using data with few classes, it is possible to detect an object regarding the few classes at the time of inference and output a detection result. It may be also considered that such a learning method is suitable for the above-described applications not requiring the detection result of the objects in fine multiple classes. However, in this case, since the number of classes of data at the time of learning is small, a detection performance of an object at the time of inference is deteriorated.

For example, a case where the object position region detection unit and the class identification unit are allowed to learn so as to detect three classes of "person", "vehicle", and "animal" from an input image is considered. In this case, for example, it is necessary to allow them to learn "bird" and "dog" as the same "animal" class. However, since there are few common characteristics between "bird" and "dog", it is difficult to allow them to learn "bird" and "dog" as the same "animal" class. For this reason, it is difficult to recognize (identify) "bird" and "dog" as the same "animal" class at the time of inference, and as a result, the detection performance of the object is deteriorated.

Then, in the applications in which the detection result of the objects in fine multiple classes is not required, there is a need for a technology for improving the detection performance of the objects having few common characteristics while enabling object detection with rough small classes. However, such technology has not yet been proposed also in Patent Literatures <NUM> to <NUM>.

The present invention has been achieved to solve the above-described problems, and an object thereof is to provide an object detection device, an object detection method, a program, and a recording medium that may improve a detection performance of an object by performing detection of a position region and object detection based on class identification with fine multiple classes and that are suitable for applications not requiring a detection result of an object in units of fine multiple classes.

According to a first solution, the present invention provides an object detection device in accordance with independent claim <NUM>. According to a second solution, the present invention provides an object detection method according to independent claim <NUM>. According to a third solution, the present invention provides a program according to claim <NUM>. Further solutions and aspects of the invention are set forth in the dependent claims, the drawings and the following description.

An object detection device according to an aspect of the present invention is provided with an object position region detection unit that detects, from an input image, a position region of an object included in the image on the basis of a first class definition in which a plurality of classes is defined in advance, a class identification unit that identifies a class out of the plurality of classes to which the object belongs on the basis of the first class definition, and an object detection result output unit that outputs a detection result of the object on the basis of a detection result of the object position region detection unit and an identification result of the class identification unit, in which the object detection result output unit outputs class information of the object as the detection result of the object on the basis of a second class definition in which a plurality of classes is defined in advance, the second class definition associated with the first class definition, and the number of classes defined in the second class definition is smaller than the number of classes defined in the first class definition.

An object detection method according to another aspect of the present invention is provided with an object position region detection step of detecting, from an input image, a position region of an object included in the image on the basis of a first class definition in which a plurality of classes is defined in advance, a class identification step of identifying a class out of the plurality of classes to which the object belongs on the basis of the first class definition, and an object detection result output step of outputting a detection result of the object on the basis of a detection result of the object position region detection step and an identification result of the class identification step, in which at the object detection result output step, class information of the object is output as the detection result of the object on the basis of a second class definition in which a plurality of classes is defined in advance, the second class definition associated with the first class definition, and the number of classes defined in the second class definition is smaller than the number of classes defined in the first class definition.

A program according to still another aspect of the present invention is a program for allowing a computer to execute the above-described object detection method.

A recording medium according to still another aspect of the present invention is a computer-readable recording medium in which the above-described program is recorded.

The number of a plurality of classes defined in a first class definition is relatively large as compared to the number of classes defined in a second class definition, and in the first class definition, a plurality of classes is finely defined. Therefore, it becomes possible to detect a position region of an object and identify a class with fine multiple classes on the basis of the first class definition. As a result, it becomes possible to perform the object detection with fine multiple classes, and improve a detection performance of the object.

The second class definition is associated and correlated with the first class definition, and the number of classes defined in the second class definition is smaller than the number of classes defined in the first class definition. For this reason, it becomes possible to output a detection result of the object in units of few classes on the basis of the second class definition. Therefore, it is possible to realize an object detection device or an object detection method suitable for applications not focusing on the detection performance in units of fine multiple classes.

An embodiment of the present invention is described as follows with reference to the drawings.

<FIG> is a block diagram schematically illustrating an entire configuration of an object detection system <NUM> according to this embodiment. The object detection system <NUM> includes an imaging unit <NUM> and an object detection device <NUM>. The imaging unit <NUM> is formed of a camera that images an object to acquire an image. The above-described image is, for example, a moving image, but may also be a still image. The number of imaging units <NUM> included in the object detection system <NUM> is not especially limited, and one or a plurality of imaging units <NUM> may be included.

The imaging unit <NUM> is communicably connected to the object detection device <NUM> via a communication line such as a wired cable or a local area network (LAN). Note that, the imaging unit <NUM> may be configured to be able to wirelessly communicate with the object detection device <NUM>. In any case, data of the image acquired by imaging by the imaging unit <NUM> is transmitted to the object detection device <NUM> via the communication line.

The object detection device <NUM> is formed of, for example, a personal computer (PC). The object detection device <NUM> includes an object position region detection unit <NUM>, a class identification unit <NUM>, an object detection result output unit <NUM>, a storage unit <NUM>, an input unit <NUM>, a display unit <NUM>, a communication unit <NUM>, a read unit <NUM>, and a control unit <NUM>. The control unit <NUM> is formed of a central processing unit (CPU) that controls an operation of each unit of the object detection device <NUM>, and operates according to an operation program stored in a program storage unit 14a of the storage unit <NUM>.

The object position region detection unit <NUM> detects, from an input image, a position region of an object included in the image on the basis of a first class definition in which a plurality of classes is defined in advance. Herein, the "input image" may be considered to be, for example, an image input from the imaging unit <NUM> to the object detection device <NUM>, but is not limited to this image; in addition, an image input from an external terminal device or server to the object detection device <NUM> via a communication line may also be considered.

The object position region detection unit <NUM> is formed of, for example, a graphics processing unit (GPU). The GPU is an arithmetic device (processor) specialized in real-time image processing. By forming the object position region detection unit <NUM> of the GPU, it becomes possible to construct a neural network capable of performing machine learning such as a convolutional neural network (CNN) in the GPU and allow this neural network to serve as the object position region detection unit <NUM>.

In this embodiment, for example, <NUM> classes are defined as the above-described first class definition. The above-described <NUM> classes are, for example, "person (with head)", "person (without head) (with head hidden)", "car", "bus", "truck", "dog", "cat", "horse", "bear", and "bird". The object position region detection unit <NUM> is allowed to learn in advance to detect a position region of an object that might belong to the class defined in the first class definition from the input image (a parameter (weight) of each node forming the CNN is set in advance). As a result, when an image is actually externally input, the object position region detection unit <NUM> may detect a position region of an object like "car", for example, from the input image on the basis of the first class definition.

The class identification unit <NUM> identifies a class out of a plurality of classes to which the object included in the input image belongs on the basis of the above-described first class definition. As is the case with the object position region detection unit <NUM>, such class identification unit <NUM> is formed of a GPU. As a result, it becomes possible to construct a neural network capable of performing machine learning such as a CNN in the GPU and allow this neural network to serve as the class identification unit <NUM>.

The class identification unit <NUM> is allowed to learn in advance to identify the class to which the object included in the input image belongs on the basis of the first class definition (a parameter (weight) of each node forming the CNN is set in advance). As a result, when an image is actually externally input, the class identification unit <NUM> may identify, on the basis of the first class definition, the class out of the classes defined in the first class definition to which the object included in the input image (including the object of which position region is already detected by the object position region detection unit <NUM>) belongs.

In this embodiment, since the class identification unit <NUM> is formed of the CNN, not only an identification result of the class to which the object belongs but also a score indicating likelihood (certainty) of the class is output from the class identification unit <NUM>. For example, information (class information with score) that the certainty that the object is "person (with head)" is <NUM> and the like is output from the class identification unit <NUM>.

The object detection result output unit <NUM> outputs a detection result of the object on the basis of a detection result of the object position region detection unit <NUM> and the identification result of the class identification unit <NUM>. For example, the object detection result output unit <NUM> outputs information of the position region of the object detected by the object position region detection unit <NUM> and information of the class identified by the class identification unit <NUM>. Such object detection result output unit <NUM> is formed of, for example, the same CPU as or a different CPU from that of the control unit <NUM>.

In this embodiment, the object detection result output unit <NUM> outputs class information of the object as the detection result of the object on the basis of a second class definition in which a plurality of classes is defined in advance.

Herein, in the above-described second class definition, for example, three classes are defined. The above-described three classes are, for example, "person", "vehicle", and "animal". Therefore, the number of classes (three) defined in the second class definition is smaller than the number of classes (<NUM>) defined in the first class definition.

In addition, "person" defined in the second class definition is a class (superordinate concept class) including "person (with head)" and "person (without head)" defined in the first class definition. In addition, "vehicle" defined in the second class definition is a class including "car", "bus", and "truck" defined in the first class definition. Furthermore, "animal" defined in the second class definition is a class including "dog", "cat", "horse", "bear", and "bird" defined in the first class definition. Therefore, it may be said that each class defined in the first class definition and each class defined in the second class definition are associated (correlated) with each other, and the first class definition and the second class definition are associated with each other.

Since the object detection result output unit <NUM> outputs the class information of the object as the detection result of the object on the basis of the second class definition, when the information that certainty that the object is "person (with head)" is <NUM> and certainty that the object is "car" is <NUM> is output from the class identification unit <NUM>, for example, this outputs the class information that the class of the above-described object is "person" (inclusive concept of "person (with head)") as the detection result of the object.

The storage unit <NUM> is a memory that stores various types of information, and formed of, for example, a hard disk, but this may further include a random access memory (RAM), a read only memory (ROM) and the like. The storage unit <NUM> includes the program storage unit 14a, an input image storage unit 14b, a class definition storage unit 14c, and a parameter storage unit 14d.

The program storage unit 14a stores the operation program for operating each unit of the object detection device <NUM>. The input image storage unit 14b temporarily stores the data of the image acquired by the imaging unit <NUM> and input to the object detection device <NUM>, for example. Note that, when image data is input from the external terminal device or server to the object detection device <NUM>, the input image storage unit 14b may temporarily store the above-described image data. The image input to the object detection device <NUM> may be directly input to the object position region detection unit <NUM> without being stored in the input image storage unit 14b.

The class definition storage unit 14c stores the above-described first class definition (for example, <NUM> classes) and second class definition (for example, three classes). Note that, the first class definition and the second class definition stored in the class definition storage unit 14c may be changed (updated) by the input unit <NUM> to be described later. The above-described object position region detection unit <NUM> may detect the position region of the object on the basis of the first class definition by accessing the class definition storage unit 14c. The class identification unit <NUM> may identify the class of the object on the basis of the first class definition by accessing the class definition storage unit 14c. Furthermore, the object detection result output unit <NUM> may output the class information of the object as the detection result of the object on the basis of the second class definition by accessing the class definition storage unit 14c.

The parameter storage unit 14d stores a parameter (weight) of each node of the CNN forming the object position region detection unit <NUM> and the class identification unit <NUM>. The above-described parameter is updated every learning of the CNN and stored in the parameter storage unit 14d.

The input unit <NUM> is formed of, for example, a keyboard, a mouse, a touch pad, a touch panel and the like, and receives various instruction inputs by a user. The display unit <NUM> is a device that displays various types of information, and is formed of, for example, a liquid crystal display device. The communication unit <NUM> is an interface for communicating with the imaging unit <NUM> and an external terminal (for example, another terminal device (including a mobile terminal such as a smartphone) and a server). The communication unit <NUM> includes an antenna, a transmission/reception circuit, a modulation circuit, a demodulation circuit and the like in addition to an input/output port. The read unit <NUM> is a device that reads information recorded in a recording medium R, and is formed of, for example, a disk drive. Note that, in a case where the recording medium R is a portable nonvolatile memory, a connection port into which a connection unit of the above-described nonvolatile memory is inserted is also included in the read unit <NUM>.

Next, operations of the object detection system <NUM> of this embodiment at the time of learning and at the time of inference (at the time of object detection) are separately described. Note that, in the following description, the above-described <NUM> classes (for example, "person (with head)", "person (without head)", "car", "bus", "truck", "dog", "cat", "horse", "bear", and "bird") are defined as the first class definition, and the above-described three classes (for example, "person", "vehicle", and "animal") are defined as the second class definition.

<FIG> is an explanatory diagram illustrating a flow of processing at the time of learning in the object detection system <NUM>. First, a data set for learning is prepared prior to the learning (S1). The above-described data set is a set of data of an image including an object and a label indicating a correct answer class of the object. Herein, the data sets including data of images of <NUM> classes are prepared on the basis of the above-described first class definition. Note that, a plurality of data sets may be prepared for each of the <NUM> classes. For example, a plurality of data sets of different images may be prepared for "person (with head)" forming one class of the <NUM> classes.

When the image of the data set for learning is input to the object detection device <NUM> (S2), the object position region detection unit <NUM> detects a position region of an object that might belong to any one of the <NUM> classes from the above-described image by neuro calculation by the CNN on the basis of the first class definition (S3). Then, the class identification unit <NUM> identifies a class to which the class of the object in the position region detected by the object position region detection unit <NUM> belongs out of the <NUM> classes defined in the first class definition by neuro calculation by the CNN (S4).

Thereafter, the control unit <NUM> updates the parameter of each node of the CNN forming the object position region detection unit <NUM> and the class identification unit <NUM> by using a back propagation method on the basis of the information output from the class identification unit <NUM> (S5). More specifically, this updates the parameter of each node of the object position region detection unit <NUM> such that the position region is detected for the object of the correct answer class (class of the first class definition) corresponding to the input image, and updates the parameter of each node of the class identification unit <NUM> such that the correct answer class defined in the first class definition is output as the identification result of the class of the object present in the position region detected by the object position region detection unit <NUM>.

<FIG> is an explanatory diagram illustrating an example of processing at the time of inference in the object detection system <NUM>. For example, when the image acquired by the imaging unit <NUM> is input to the object detection device <NUM> (S11), the object position region detection unit <NUM> detects a position region of an object that might belong to any one of the <NUM> classes from the above-described image on the basis of the first class definition by neuro calculation by the CNN (S12; object position region detection step).

Next, the class identification unit <NUM> identifies a class to which the class of the object in the position region detected by the object position region detection unit <NUM> belongs out of the <NUM> classes defined in the first class definition by neuro calculation by the CNN, and outputs information of the identified class with score (S13; class identification step).

The object detection result output unit <NUM> selects the class of the object with the highest score from the class information output from the class identification unit <NUM>, and determines the class including the selected class on the basis of the second class definition. Then, the object detection result output unit <NUM> outputs information of the determined class and information of the position region detected by the object position region detection unit <NUM> as the detection result of the object (S14; object detection result output step). The detection result of the object (information of the position region and class information) output from the object detection result output unit <NUM> may be displayed on the display unit <NUM>, for example, or may be transmitted to outside via the communication unit <NUM>.

In this embodiment, the number of classes (for example, <NUM>) defined in the first class definition is larger than the number of classes (for example, three) defined in the second class definition, and in the first class definition, a plurality of classes is finely defined. For this reason, on the basis of the first class definition, the object position region detection unit <NUM> may detect the position regions of the respective objects (for example, "bird" and" dog") having few common characteristics although they are common as, for example, same "animal" while distinguishing them from each other. Then, the class identification unit <NUM> may identify the objects (for example, "bird" and "dog") having few common characteristics in different classes on the basis of the first class definition. Therefore, the object may be detected with fine multiple classes, and a detection performance of the object may be improved.

When outputting the detection result of the object on the basis of the detection result of the object position region detection unit <NUM> and the identification result of the class identification unit <NUM>, the object detection result output unit <NUM> outputs the class information of the object on the basis of the second class definition associated with the first class definition. Since the number of a plurality of classes defined in the second class definition is smaller than the number of classes defined in the first class definition, the object detection result output unit <NUM> may output, for example, information of a class in which "bird" and "dog" are put together (for example, information of "animal"). Therefore, it is possible to output the detection result of the object in units of classes fewer than those at the time of detection of the position region of the object and at the time of class identification, so that it is possible to realize the object detection device <NUM> and an object detection method suitable for uses and applications not focusing on the detection performance in units of fine multiple classes. For example, the object detection device <NUM> and the object detection method of this embodiment are suitable for uses and applications for performing investigation and analysis intended only to count passed objects and do not require detailed identification of the above-described objects in addition to monitoring and statistical investigation using an image captured by a camera installed outdoors.

In this embodiment, "person (with head)" and "person (without head)" in the first class definition are subordinate concept classes of "person" defined in the second class definition, that is, the classes obtained by subdividing "person" described above. Similarly, "car", "bus", and "truck" in the first class definition are the classes obtained by subdividing "vehicle" defined in the second class definition. Furthermore, "dog", "cat", "horse", "bear", and "bird" in the first class definition are classes obtained by subdividing "animal" defined in the second class definition.

In this manner, since the first class definition is defined by subdividing each class defined in the second class definition, it is possible to detect the object with fine multiple classes on the basis of the first class definition. As a result, the detection performance of the object may be reliably improved. The object detection result output unit <NUM> may put two or more classes (for example, "dog" and "bird") identified on the basis of the first class definition together into one class (for example, "animal") defined in the second class definition, and output this information as the class information. Therefore, the object detection result output unit <NUM> may reliably output the detection result of the objects in units of fewer classes than those at the time of detection of the position region of the object and at the time of class identification.

In addition, "person (with head)" and "person (without head)" in the first class definition have a common characteristic as "person". In addition, "car", "bus", and "truck" in the first class definition have a common characteristic as "vehicle". Furthermore, "dog", "cat", "horse", "bear", and "bird" in the first class definition have a common characteristic as "animal".

From this, it may be said that the first class definition is defined by subdividing each class defined in the second class definition into different classes having the common characteristic. In this case, the object position region detection unit <NUM> may detect the respective position regions of the objects that might belong to different classes having the common characteristic while distinguishing them from each other on the basis of the first class definition. The class identification unit <NUM> may separately identify different classes having the common characteristic on the basis of the first class definition. Therefore, it becomes possible to detect the objects in a distinguishing manner even between the different classes having the common characteristic, and reliably improve the detection performance of the object.

In this embodiment, the object position region detection unit <NUM> and the class identification unit <NUM> are formed of the neural network. In this case, by allowing the object position region detection unit <NUM> and the class identification unit <NUM> to perform machine learning, it becomes possible to detect the position region of the object by neuro calculation on the basis of the input image, and to identify the class of the object by neuro calculation. By updating the parameter (weight of each node) of the neural network by the machine learning, it becomes possible to improve detection accuracy of the position region of the object (detection accuracy of presence or absence of the object) and accuracy of the class identification.

In this embodiment, the class identification unit <NUM> identifies the class after the object position region detection unit <NUM> detects the position region (refer to <FIG>). In this case, since the class identification unit <NUM> may identify the class of the object the position region of which is detected by the object position region detection unit <NUM> (since the class may be identified only for something like the object), the detection performance of the object may be further improved.

<FIG> is an explanatory diagram schematically illustrating a hierarchical structure of each class defined in the second class definition. The first class definition may be defined on the basis of the hierarchical structure of each class defined in the second class definition. In the drawing, an example is illustrated in which the above-described hierarchical structure is realized by subdividing each class defined in the second class definition on the basis of the identification result of the class identification unit <NUM> at the time of learning, and each class in the first class definition is defined on the basis of the hierarchical structure.

More specifically, assume that each class defined in the second class definition is two classes of "person" and "vehicle". For each of these classes, the data set for learning (image + correct answer label) is prepared. At that time, a plurality of data sets of different images is prepared for each class. Then, the object position region detection unit <NUM> and the class identification unit <NUM> are allowed to learn by the above-described method. That is, for example, the image of the data set of "person" is input, the object position region detection unit <NUM> detects the position region of the object and the class identification unit <NUM> performs the class identification, and the identification result (score of the identified class) is output. Then, the parameter of each node of the object position region detection unit <NUM> and the class identification unit <NUM> is updated by using the back propagation method.

At that time, in the data set of "person", in a case where the class identified by the class identification unit <NUM> (class with the score equal to or higher than a threshold) matches the class (for example, "person") indicated by the correct answer label, it is determined that the object detection is succeeded, and the class identified by the class identification unit <NUM> (for example, "person") is left as it is. In contrast, in the data set of "person", in a case where the class identified by the class identification unit <NUM> (class with the score equal to or higher than a threshold) does not match the class (for example, "person") indicated by the correct answer label, it is determined that the object detection is failed, and the class identified by the class identification unit <NUM> is set as a new class (for example, "person <NUM>"). Then, the parameter of each node of the object position region detection unit <NUM> and the class identification unit <NUM> is updated by the back propagation method so that the object included in the image of the used data set (the image in which the detection of the object is failed) is identified as "person <NUM>".

Next, the image of the data set of "person <NUM>" is input, the object position region detection unit <NUM> detects the position region of the object and the class identification unit <NUM> performs the class identification, the identification result (score of the identified class) is output, and the processing similar to that described above is performed. That is, in a case where the class identified by the class identification unit <NUM> (class with the score equal to or higher than a threshold) matches the class (for example, "person <NUM>") indicated by the correct answer label, it is determined that the object detection is succeeded, and the class identified by the class identification unit <NUM> (for example, "person <NUM>") is left as it is. In contrast, in the data set of "person <NUM>", in a case where the class identified by the class identification unit <NUM> (class with the score equal to or higher than a threshold) does not match the class (for example, "person <NUM>") indicated by the correct answer label, it is determined that the object detection is failed, and the class identified by the class identification unit <NUM> is set as a new class (for example, "person <NUM>"). Then, the parameter of each node of the object position region detection unit <NUM> and the class identification unit <NUM> is updated by the back propagation method so that the object included in the image of the used data set (the image in which the detection of the object is failed) is identified as "person <NUM>".

By the above-described processing, the class of "person" is subdivided into the classes of "person" and "person <NUM>" on the basis of the identification result by the class identification unit <NUM>, and the class of "person <NUM>" is further subdivided into the classes of "person <NUM>" and "person <NUM>" on the basis of the identification result by the class identification unit <NUM>. As a result, a hierarchical structure in which "person" is in an uppermost layer is realized. That is, on the basis of the identification result by the class identification unit <NUM>, the hierarchical structure in which "person" in the uppermost layer class is subdivided into lower layer classes ("person", "person <NUM>", and "person <NUM>") is obtained.

Note that, as for "vehicle" also, the object position region detection unit <NUM> and the class identification unit <NUM> are allowed to learn in the manner similar to that described above by using the data set of "vehicle", so that a hierarchical structure of "vehicle" may be obtained on the basis of the identification result by the class identification unit <NUM>. That is, it is possible to obtain the hierarchical structure in which "vehicle" in the uppermost layer class is subdivided into lower layer classes ("vehicle", "vehicle <NUM>", and "vehicle <NUM>") on the basis of the identification result by the class identification unit <NUM>.

Therefore, by defining "person" and "vehicle" as each class in the second class definition, it is possible to define the first class definition on the basis of the hierarchical structures of "person" and "vehicle" defined in the second class definition. That is, the classes belonging to the layers other than the uppermost layers of the hierarchical structures of the two classes ("person" and "vehicle") defined in the second class definition, that is, "person", "person <NUM>", "person <NUM>", "vehicle", "vehicle <NUM>", and "vehicle <NUM>" (six classes) may be defined as the first class definition.

In this manner, by defining the first class definition on the basis of the hierarchical structure of each class defined in the second class definition, it becomes possible to reliably set the first class definition finely with multiple classes associated with the second class definition. As a result, the detection of the position region of the object and the class identification by the object position region detection unit <NUM> and the class identification unit <NUM>, respectively, may be reliably performed with fine multiple classes on the basis of the first class definition. The object detection result output unit <NUM> may reliably reduce the number of classes output as the detection result of the object by outputting information of the uppermost layer class of the above-described hierarchical structure on the basis of the second class definition.

As described above, by subdividing each class defined in the second class definition on the basis of the identification result by the class identification unit <NUM> at the time of learning and defining the first class definition, it becomes possible to reliably improve the identification accuracy when the class identification unit <NUM> identifies the class on the basis of the first class definition at the time of inference. That is, in the above-described example, at the time of inference, it is possible to reliably and accurately identify which of "person", "person <NUM>", and "person <NUM>" the person included in the input image is, or which of "vehicle", "vehicle <NUM>", and "vehicle <NUM>" the vehicle included in the input image is.

As described above, by defining the first class definition by subdividing each class defined in the second class definition into a plurality of classes including the class (for example, "person" and "person <NUM>") for which the identification at the time of learning is succeeded and the class (for example, "person <NUM>") newly assigned after the failure in identification at the time of learning, it becomes possible to determine (identify) whether the object belongs to the class (for example, "person <NUM>") on the basis of the first class definition at the time of inference also for the class (for example, "person <NUM>") for which the identification is failed at the time of learning.

<FIG> is an explanatory diagram illustrating another example of processing at the time of inference in the object detection system <NUM>. <FIG> is similar to <FIG> except that the class identification unit <NUM> identifies the class of the object included in the image simultaneously with the detection of the position region by the object position region detection unit <NUM>. In this configuration, the object for which the class identification unit <NUM> identifies the class is not an object present in a specific position region in the image but an object present somewhere in the input image; however, in this case also, it becomes possible to identify the class of the object (object the position region of which is not specified) located somewhere in the input image at the time of inference by allowing the class identification unit <NUM> to perform machine learning in advance.

In this manner, by simultaneously performing the detection of the position region by the object position region detection unit <NUM> and the identification of the class of the object by the class identification unit <NUM>, it becomes possible to perform quick object detection (output of the detection result) as compared with a case of continuously performing the processing as illustrated in <FIG>.

The object detection device <NUM> of this embodiment described above may be formed of, for example, a computer (PC) in which a predetermined program (application software) is installed. By reading and executing the above-described program by the computer (for example, the control unit <NUM> as the CPU), it becomes possible to operate each unit of the object detection device <NUM> to execute each processing (each step) described above. Such program is acquired by being externally downloaded via, for example, a network and stored in the program storage unit 14a. The above-described program may be recorded in a computer-readable recording medium R such as a compact disk-read only memory (CD-ROM), for example, and the computer may read the above-described program from the recording medium R and store the same in the program storage unit 14a.

Claim 1:
An object detection device (<NUM>) comprising:
an object position region detection unit (<NUM>) configured to detect, from an input image, a position region of an object included in the image on the basis of a first class definition in which a plurality of classes is defined in advance;
a class identification unit (<NUM>) configured to identify a class out of the plurality of classes to which the object belongs on the basis of the first class definition; and
an object detection result output unit (<NUM>) configured to output a detection result of the object on the basis of a detection result of the object position region detection unit (<NUM>) and an identification result of the class identification unit (<NUM>), wherein
the object detection result output unit (<NUM>) is configured to output class information of the object as the detection result of the object on the basis of a second class definition in which a plurality of classes is defined in advance, the second class definition associated with the first class definition, and
the number of classes defined in the second class definition is smaller than the number of classes defined in the first class definition, and
wherein the first class definition is defined by subdividing each class defined in the second class definition.