Verification device, verification method, and verification program

A verification device includes an acquisition unit that acquires image data which is an object to be verified, a classification unit that includes a plurality of classifiers caused to learn the image data acquired by the acquisition unit, using a plurality of different pieces of learning data, with respect to a target event to obtain values indicating a possibility corresponding to the event, and a specification unit that specifies a state of the object to be verified which is the image acquired by the acquisition unit from the values obtained by the plurality of classifiers.

TECHNICAL FIELD

This disclosure relates to a verification device, a verification method, and a verification program and, more particularly, to a verification device using machine learning, a verification method, and a verification program.

BACKGROUND

Hitherto, images have been used for a variety of verifications. For example, a lesion and the like have been confirmed using images regarding a human's features. In addition, the presence or absence of foreign matter mixed during the manufacture of a product and a defect have also been confirmed using an image of the product. Further, a defect of a semiconductor part disposed on a semiconductor substrate has also been confirmed using an image of the manufactured semiconductor substrate. When such confirmation is performed, a machine learning technique called “deep learning” has also been used as auxiliary means therefor to avoid oversight.

As a method using machine learning, a method using a classifier caused to learn using learning data is known. For example, when a “lesion” being a change in mind and body which is generated due to a disease is verified from image data by using machine learning, a plurality of pieces of image data of a “positive example” having a target lesion and a plurality of pieces of image data of a “negative example” having no target lesion are prepared and set to be learning data.

At that time, the accuracy of the classifier depends on learning data. However, the accuracy does not necessarily become higher as the number of pieces of learning data increases. For example, an increase in the number of pieces of data may results in a state called “over-learning” in which the accuracy in another data decreases due to excessive application to given data. When the accuracy of the classifier is improved by avoiding the over-learning, a variety of tunings are required.

Examples of the related art include Japanese Unexamined Patent Application Publication No. 2015-176283, Japanese Unexamined Patent Application Publication No. 2015-176169, Japanese Unexamined Patent Application Publication No. 2014-135092, Japanese Unexamined Patent Application Publication No. 2005-115525, and https://blogs.nvidia.com/blog/2016/05/10/deep-learning-chest-x-rays/ (searched on Jul. 22, 2016).

As described above, in a case where a classifier is used, a variety of adjustments such as tuning are required, and thus it is difficult to easily obtain a highly accurate result. Therefore, for example, it is difficult for a user or the like who is not familiar with machine learning to easily obtain a highly accurate result by using machine learning.

It could therefore be helpful to provide a verification device capable of easily obtaining a highly accurate verification result by using machine learning.

SUMMARY

I thus provide a verification device including an acquisition unit that acquires image data which is an object to be verified, a classification unit that includes a plurality of classifiers caused to learn the image data acquired by the acquisition unit, using a plurality of different pieces of learning data with respect to a target event, to obtain values indicating a possibility corresponding to the event, and a specification unit that specifies a state of the object to be verified which is the image acquired by the acquisition unit from the values obtained by the plurality of classifiers.

According to a verification device, a verification method, and a verification program, it is possible to easily obtain a highly accurate verification result by using machine learning.

I also provide a verification method including an acquisition step of acquiring image data which is an object to be verified; a classification step of obtaining values indicating a possibility corresponding to a target event by a plurality of classifiers caused to learn the acquired image data, using a plurality of different pieces of learning date with respect to the target event; and a specification step of specifying a state of the object to be verified which is the acquired image from the obtained values.

I further provide a verification program causing a computer to execute functions of acquiring image data which is an object to be verified; causing a plurality of classifiers to learn the acquired image data, using a plurality of different pieces of learning data with respect to a target event, and to obtain values indicating a possibility corresponding to the target event; and specifying a state of the object to be verified which is the acquired image from the values obtained by the plurality of classifiers.

DETAILED DESCRIPTION

My verification device verifies an event corresponding to an object to be verified, by using image data which is the acquired object to be verified. Hereinafter, a verification device will be described with reference to the accompanying drawings. In the following description, the same components will be denoted by the same reference numerals and signs, and a description thereof will not be repeated. Meanwhile, in the following description, a description will be given on the assumption that the possibility of presence or absence of a lesion is verified.

As illustrated inFIG. 1, a verification device1includes an acquisition unit101, a classification unit102, and a specification unit103. The acquisition unit101acquires image data which is an object to be verified. The classification unit102includes a plurality of classifiers caused to learn the image data acquired by the acquisition unit101, using a plurality of different pieces of learning data with respect to a target event, to obtain values indicating a possibility corresponding to the event. The specification unit103specifies the state of the object to be verified which is the image acquired by the acquisition unit101from the values obtained by the plurality of classifiers.

Verification Device

The verification device1is an information processing device such as a personal computer including a central processing unit (CPU)10, a storage device11, an input unit12, an output unit13, and a communication I/F14as illustrated inFIG. 2. The storage device11includes a verification program P, and the verification program P is executed so that the CPU10executes processes as the acquisition unit101, the classification unit102, the specification unit103, a reception unit104, and a learning unit105.

In addition, the storage device11can store learning data112and change condition data113together with the verification program P.

The learning data112includes image data when there is no abnormality and image data when there is an abnormality which are specified as a target event. When an object to be verified by the verification device1relates to the presence or absence of a lesion, the learning data112includes image data (negative example data) when there is no predetermined lesion generated in an animal or a human and image data (positive example data) when there is a lesion which are specified as a target event.

In the learning data112, a plurality of data sets are specified as learning data which is a positive example. For example, the plurality of pieces of positive example data are divided into groups for each predetermined number, and the groups can be set to be one data set. Specifically, when there are 1000 pieces of positive example data and 100 pieces of data are set to be one group, the pieces of data are divided into 10 groups. At this time, 100 pieces of data extracted randomly may be set to be one data set (group), and the repetition thereof may create a plurality of data sets (groups).

In the learning data112, for example, all pieces of image data other than image data having a target lesion can be set to be pieces of negative example data. In addition, for example, image data having no lesion may be set to be negative example data. In addition, data having another lesion easily mistaken for a target lesion may be set to be negative example data. Meanwhile, learning data which is a negative example may also use a plurality of different data sets.

The change condition data113is data specifying a change condition for a weight value used by the specification unit103depending on an object to be verified. In the classification unit102, the plurality of classifiers obtain values of classification results, but the weights of the values obtained by the respective classifiers may be different from each other. At this time, performing weighting is to obtain a weight value. In addition, it may be preferable to change the value of weighting depending on a method of acquiring target data. For example, when image data acquired by the acquisition unit101is a fundus image, a lesion appearing in the image data may be different depending on a method of acquiring the image, specifically, in which cross-section the image is acquired. For example, as illustrated inFIG. 3A, different results are obtained in cross-sections taken along line A-A and B-B of a fundus image of a certain test subject. For example, the image obtained on the basis of A-A is as illustrated inFIG. 3B, and the image obtained on the basis of B-B is as illustrated inFIG. 3C. In this case, determination results may be different from each other depending on the method of acquiring an image. Therefore, the images are data regarding a condition for changing a weight value depending on the type of image data, that is, a method of acquiring image data.

The acquisition unit101acquires image data which is an object to be verified. For example, the acquisition unit101may acquire the image data from an image capture device (not shown), or may acquire the image data from another information processing device. In addition, the wording “image data” as used herein is not limited to two-dimensional image data and three-dimensional image data, and may be data regarding an analysis result obtained from the two-dimensional image data or the three-dimensional image data.

The classification unit102includes a plurality of classifiers caused to learn the image data acquired by the acquisition unit101, using a plurality of different pieces of learning data with respect to a target event, to obtain values indicating a possibility corresponding to the event. The value included in each classifier is a value indicating a possibility corresponding to the target event which is set to be the degree of certainty in a general classifier. The classification unit102includes at least one classifier caused to learn using the image data when there is no abnormality, and a plurality of classifiers caused to learn using the image data when there is an abnormality. When an object to be verified by the verification device1relates to the presence or absence of a lesion, the classification unit102includes at least one classifier caused to learn using learning data when there is no lesion, and a plurality of classifiers caused to learn using learning data when there is a lesion.

The classifier used in the classification unit102is a classifier such as a simple Bayesian classifier, which is specified by machine learning, and classifies events corresponding to target data newly given, by using learning data (teacher data) which is given in advance.

As described above, a plurality of different data sets are specified with respect to a target lesion in the learning data112. Therefore, the classification unit102includes a plurality of classification units with respect to each data set. In addition, my method is not limited to one type of lesion, and it is possible to verify the presence or absence of a plurality of types of lesions by including pieces of data regarding the plurality of types of lesions as the learning data112.

The specification unit103specifies the state of the object to be verified which is the image acquired by the acquisition unit101from the values obtained by the plurality of classifiers. The specification unit103specifies the possibility of presence or absence of an abnormality in an object to be verified as the state of the object to be verified by using a calculation expression (to be described later by using, for example, Expression (1)) which comprehensively considering the values obtained by the respective classifiers. When the object to be verified by the verification device1relates to the presence or absence of a lesion, the specification unit103specifies the possibility of presence or absence of a specific lesion as the state of an animal or a human which is an object to be verified.

The specification unit103calculates a weight value to weight the values in accordance with the values obtained by the plurality of classifiers to specify the state of the object to be verified which is the image acquired by the acquisition unit101from values obtained by weighting the values.

For example, to obtain the degree of certainty when a certain lesion1is generated, a value (the degree of certainty) a=C1,N(T) and a value (degree of certainty) b=C1,other(T) are used as weight values, as shown in Expression (1). a=C1,N(T) is obtained by a classifier in which image data in a case of the lesion1is used as a positive example and image data in a normal case is used as a negative example, and b=C1,other(T) is obtained by a classifier in which image data when the lesion1is used as a positive example and image data when other than the lesion1is used as a negative example. Meanwhile, C1,2(T) is a value (degree of certainty) which is obtained by a classifier in which image data when the lesion1is used as a positive example and image data when a lesion2is used as a negative example.
C1,N(T)+((a+b)/2)·C1,2(T)  (1)

The above-described example of Expression (1) is an example in which “a weight value of the value obtained by the classifier in which the image data when the lesion1is used as a positive example and the image data when the lesion2is used as a negative example” is calculated using “the value obtained by the classifier in which the image data when the lesion1is used as a positive example and normal image data is used as a negative example” and “the value obtained by the classifier in which the image data when the lesion1is used as a positive example and the image data when other than the lesion1is used as a negative example.”

In addition, the specification unit103extracts a change condition of a weight value associated with the type of image data acquired by the acquisition unit101from change condition data in which the types of image data and change conditions of weight values used for the weighting are associated with each other and which is stored in a change condition storage unit, and changes the weight value by using the extracted change condition, thereby specifying an event corresponding to the object to be verified which is the image acquired by the acquisition unit101by using the changed weight value. The specification unit103specifies the event corresponding to the object to be verified by using target data input to the reception unit104, in addition to a specification result obtained on the basis of the image data acquired by the acquisition unit101.

The reception unit104receives an input of target data acquired by another method with respect to the object to be verified together with the image data acquired by the acquisition unit101. For example, when the verification device1verifies a lesion of an animal or a human, the target data is image data, obtained by an acquisition device, which is different from the image data acquired by the acquisition unit101, data obtained as a result of blood collecting, data regarding the progress of a lesion so far, data regarding the background of treatment and the like. Specifically, when the image data acquired by the acquisition unit101is data regarding a fundus image, the reception unit104receives an input of data of another CT image or data obtained as a result of blood collecting.

The learning unit105causes the plurality of classifiers included in the classification unit102to learn by machine learning by using pieces of learning data corresponding to the respective classifiers.

Meanwhile, the number of types of image data acquired by the acquisition unit101may be two or more. For example, image data obtained by capturing a cross-section (“first cross-section”) in a certain direction and image data obtained by capturing a cross-section (“second cross-section” different from the first cross-section) in another direction may be acquired.

When the first cross-section and the second cross-section are present, the classification unit102may classify the image data of the first cross-section and the image data of the second cross-section by the same classifier, and may perform classification by different classifiers learned using separate pieces of learning data112. Meanwhile, when the first cross-section and the second cross-section are used, the learning data112includes a plurality of different data sets with respect to positive example data for the first cross-section and also includes a plurality of different data sets with respect to negative example data. In addition, the learning data122includes a plurality of different data sets with respect to positive example data for the second cross-section and also includes a plurality of different data sets with respect to negative example data.

Meanwhile, in the example illustrated inFIG. 2, one verification device1includes the acquisition unit101, the classification unit102, the specification unit103, the reception unit104, the learning unit105, the verification program P, the learning data112, and the change condition data113, but is not limited thereto. For example, the verification device1may be constituted of a plurality of information processing devices. In addition, for example, the pieces of data112and113are stored in an external storage device connected to the verification device1, and the verification device1may read out the pieces of data112and113from the external storage device.

Verification Method

Hereinafter, processing in the verification device1will be described with reference to a flow chart illustrated inFIG. 4. In the verification device1, the classifiers of the classification unit102are caused to learn by the learning unit105using the learning data112stored in the storage device11in advance (S1). The classifiers are caused to learn using learning data of different data sets in the learning data112.

When verification is performed, the acquisition unit101acquire image data which is an object to be verified (S2).

Thereafter, each of the classifiers of the classification unit102obtains a value indicating a possibility corresponding a target event with respect to the image data acquired in step S2(S3).

Subsequently, the specification unit103calculates a weight value used when specifying the state of the object to be verified which is the image acquired in step S2from the values obtained in step S3(S4).

In addition, the specification unit103obtains a value for specifying the state of the object to be verified by using the values obtained in step S3and the weight value obtained in step S4to specify the state of the object to be verified (S5).

Specific Example

A classifier caused to learn using data, which is sampled from all other pieces of data regarding a lesion D, that is, data having no lesion D seen therein, as negative example data is set to be a “classifier I.” Similarly, a classifier using data determined to be normal data as negative example data is set to be a “classifier II.” In addition, a classifier using data regarding another lesion D′ as negative example data is set to be a “classifier III.”

A plurality of classifiers I and a plurality of classifiers II are prepared with respect to the lesion D. In addition, when the lesion D′, a lesion D″, and the like hardly distinguishable from the lesion D are present, the classifier III is also prepared.

The specification unit103may output a plurality of candidates as a specification result of the state of the object to be verified. At this time, the specification unit103may output the candidates together with the value (degree of certainty) obtained by the classification unit102, as the specification result. Further, the specification unit103may output identification information (for example, a classifier name) of a classifier obtaining a value equal to or greater than a threshold value and a list of values obtained by the classifier, as the specification result.

Meanwhile, the specification unit103may not specify the state of the object to be verified when all values obtained by the classification unit102do not reach the threshold value. Meanwhile, in the classification unit102, the classifier I and the classifier II may obtain values in a first step. As a result, when there is another confused lesion, the classifier III may obtain a value. In this case, the verification device1may include the classifier III that performs classification into the lesion D and the other plurality of lesions D′ and D″ and the like.

FIG. 5Ais an image illustrating distributions on a plane in a normal case (N) and when there is a lesion (D1to D7). In addition, D1to D7represent a difference in the type of lesion. Meanwhile, X inFIG. 5Aspecifies the position of data X acquired by the acquisition unit101.

A certain classifier performs classification into normal and when there is a lesion, as illustrated inFIG. 5B. In addition, a certain classifier I1performs classification into D1, as illustrated inFIG. 5C. Further, another classifier I2also performs classification into D1, as illustrated inFIG. 5D. Both the classifier I1and the classifier I2are classifiers for determining whether being the lesion D1, but these classifiers use different data sets as learning data. Therefore, it is possible to improve the accuracy of verification in the verification device1by combining a result of the classifier I1and a result of the classifier I2with each other. For example, as illustrated inFIG. 5C, a degree of certainty c1that the data X obtained when using the classifier I1is the lesion D1is different from a degree of certainty c2that the data X obtained when using the classifier I2is the lesion D1. That is, it is possible to improve the accuracy of verification in the verification device1by using the plurality of classifiers I1and I2with respect to the same lesion D1.

In addition, it is possible to further improve the accuracy of verification in the verification device1by using the classifier II illustrated inFIG. 6Aand the classifier III illustrated inFIG. 6Band determining a combination of values obtained by all of the classifiers with each other as illustrated inFIG. 6C. Specifically, the classifier II illustrated inFIG. 6Ais a classifier that performs only classification into normal and D1and does not consider the other pieces of data. In addition, the classifier III illustrated inFIG. 6Bis a classifier that performs only classification into D1and D6and does not consider the other pieces of data.

For example, when the classifier I1obtains a result that the degree of certainty c1of being D1is 0.9 and the classifier III obtains a result that a degree of certainty c4of being D1is 0.9, the specification unit103can output a result that the possibility of being D1is approximately 0.9.

On the other hand, when the classifier I1obtains a result that the degree of certainty c1of being D1is 0.1 and the classifier III obtains a result that the degree of certainty c4of being D1is 0.9, the specification unit103does not output a result that the possibility of being D1is approximately 0.9 by decreasing the weight of the degree of certainty c4obtained by the classifier III.

Determination Example

For example, the classification unit102obtains degrees of certainty by using seven classifiers with respect to a certain image, as follows:A degree of certainty based on a classifier ERM-2: 0.96A degree of certainty based on a classifier ERM-N: 0.95A degree of certainty based on a classifier abnormal: 0.94A degree of certainty based on a classifier ERM-1: 0.93A degree of certainty based on a classifier wetAMD-3: 0.67A degree of certainty based on a classifier wetAMD-1: 0.54.

In addition, the specification unit103specifies that the image relates to “ERM,” as a comprehensive result based on the degrees of certainty obtained by the plurality of classifiers. “ERM” and “wetAMD” are types of lesions. In addition, the classifier ERM-2, the classifier ERM-N, and the classifier ERM-1are classifiers that perform classification with respect to “ERM.” Further, the classifier wetAMD-3and the classifier wetAMD-1are classifiers that perform classification with respect to “wetAMD.” In addition, the classifier abnormal is a classifier that performs classification with respect to the presence or absence of a lesion regardless of the type of lesion.

In the degrees of certainty based on the above-described plurality of classifiers, there is a tendency for the degree of certainty regarding “ERM” to be high, and thus the specification unit103specifies “ERM.”

In this manner, the verification device1adjusts the degrees of certainty obtained by the plurality of classifiers by a weight value using the degrees of certainty, and finally specifies the state of the object to be verified.

In the above description, an example in which the verification device1verifies the possibility of presence or absence of a lesion has been described, but this disclosure is not limited thereto. For example, an image of a product can be used for the verification of presence or absence of foreign matter mixed during the manufacture of the product or the verification of a defect. In addition, an image of a semiconductor substrate manufactured can be used for the verification of a defect of a semiconductor part disposed on the substrate.

As described above, in the verification device1, a plurality of classifiers caused to learn using different pieces of learning data are used with respect to one event. In addition, in the verification device1, a final result is obtained by combining results of the plurality of classifiers with each other. Thereby, it is possible to decrease the probability of a target event being overlooked. At this time, a large number of pieces of learning data are used as a plurality of data sets, and thus it is possible to prevent a bad effect due to over-learning.

Further, in the verification device1, it is possible to verify not only one type of event, but also a plurality of types of events by combining classifiers caused to learn using learning data of a plurality of events with each other.

In addition, verification processing is realized by the CPU of the verification device1executing the verification program P and the like, but may be realized by a logic circuit (hardware) or a dedicated circuit which is formed in an integrated circuit (IC) chip, a large scale integration (LSI) or the like of the verification device1. In addition, these circuits may be realized by one or a plurality of integrated circuits, or the functions of the above-described plurality of functional units may be realized by one integrated circuit. The LSI may also be referred to as a VLSI, a super LSI, a ultra LSI, or the like by a difference in the degree of integration. That is, as illustrated inFIG. 7, the verification device1may be constituted by an acquisition circuit101a, a classification circuit102a, a specification circuit103a, a reception circuit104a, and a learning circuit105a, and the functions thereof are the same as those of the above-described units having the same names.

In addition, the verification program P may be recorded in a CPU-readable recording medium. Examples of the recording medium include a “non-transitory tangible medium” such as a tape, a disk, a card, a semiconductor memory, and a programmable logic circuit. In addition, the retrieval program may be supplied to the processor through any transmission medium (a communication network, a broadcast wave, or the like) which allows the retrieval program to be transmitted. My method can also be realized in the form of a data signal in which the video display program is embodied through electronic transmission and which is embedded in a carrier wave.

Meanwhile, the verification program P can be mounted using a script language such as ActionScript, JavaScript (registered trademark), Python, or Ruby, a compiler language such as C, C++, C#, Objective-C, or Java (registered trademark) or the like.