MACHINE LEARNING METHOD AND MACHINE LEARNING DEVICE FOR ELIMINATING SPURIOUS CORRELATION

A machine learning method includes steps of: obtaining, by a processor, a model parameter from a memory, and performing, by a processor, a classification model according to the model parameter, wherein the classification model comprises a plurality of neural network structural layers; calculating, by the processor, a first loss and a second loss according to a plurality of training samples, wherein the first loss corresponds to an output layer of the plurality of neural network structural layers, and the second loss corresponds to one, which is before the output layer, of the plurality of neural network structural layers; and performing, by the processor, a plurality of updating operations for the model parameter according to the first loss and the second loss to train the classification model.

BACKGROUND

Field of Invention

The present invention relates to a machine learning technology. More particularly, the present invention relates to a machine learning technology for eliminating spurious correlation.

Description of Related Art

Technologies such as machine learning and neural networks are widely used in a technical field of artificial intelligence. One of the important applications of artificial intelligence is to identify objects (such as human faces, vehicle license plates, etc.) or predict data (such as stock prediction, medical treatment prediction, etc.). The object detection and the data prediction can be realized through feature extraction and feature classification.

However, spurious correlation usually happens between features for the feature extraction and the feature classification, and the spurious correlation always causes that prediction accuracy of the object detection and the data prediction decreases.

SUMMARY

The disclosure provides a machine learning method, which includes following steps: obtaining, by a processor, a model parameter from a memory, and performing, by a processor, a classification model according to the model parameter, wherein the classification model comprises a plurality of neural network structural layers; calculating, by the processor, a first loss and a second loss according to a plurality of training samples, wherein the first loss corresponds to an output layer of the plurality of neural network structural layers, and the second loss corresponds to one, which is before the output layer, of the plurality of neural network structural layers; and performing, by the processor, a plurality of updating operations for the model parameter according to the first loss and the second loss to train the classification model.

The disclosure provides a machine learning device, which includes a memory and a processor. The memory is configured for storing a plurality of instructions and a model parameter; a processor is coupled with the memory. The processor is configured to run a classification model, and is configured to execute the instructions to: obtain the model parameter from the memory, and perform a classification model according to the model parameter, wherein the classification model comprises a plurality of neural network structural layers; calculate a first loss corresponding to an output layer of the plurality of neural network structural layers, and calculating a second loss corresponding to one, which is before the output layer, of the plurality of neural network structural layers; and perform a plurality of updating operations for a model parameter of the classification model according to the first loss and the second loss to train the classification model.

DETAILED DESCRIPTION

Reference is made toFIG. 1, which is a schematic diagram illustrating a machine learning device according to an embodiment of the disclosure. The machine learning device100includes a processor110and a memory120. The processor110is coupled with the memory120.

In some embodiments, the machine learning device100can be established by a computer, a server or a processing center. In some embodiments, the processor110can be realized by a central processing unit or a computing unit. In some embodiments, the memory120can be realized by a flash memory, a read-only memory (ROM), a hard disk or any equivalent storage component.

In some embodiments, the machine learning device100is not limited to include the processor110and the memory120. The machine learning device100can further include other components required to operating the machine learning device100in various applications. For example, the machine learning device100can further include an output interface (e.g., a display panel for displaying information), an input interface (e.g., a touch panel, a keyboard, a microphone, a scanner or a flash memory reader) and a communication circuit (e.g., a WiFi communication module, a Bluetooth communication module, a wireless telecommunication module, etc.).

As shown inFIG. 1, the processor110is configured to run a classification model111based on corresponding software/firmware instructions stored in the memory120.

In some embodiments, the classification model111can classify input data, for example, detecting that an input image contains vehicles, faces, license plates, text, totems, or other image-feature objects, or predicting input stock data being rising or falling in the future. The classification model111is configured to generate a corresponding label according to a classification result. It should be noted that the classification model111will refer to a model parameter MP while performing classification operations.

As shown inFIG. 1, the memory120is configured to store the model parameter MP. In some embodiments, the model parameter MP includes multiple weight parameter contents.

In this embodiment, the classification model111includes multiple neural network structural layers. In some embodiments, each one of the neural network structural layers corresponds to one weight parameter content (configured to determine the operation of one neural network structural layer) among the model parameter MP. On the other hand, each one of the neural network structural layers of the classification model111corresponds to the weight parameter content independent from others. In other words, each one of the neural network structural layers corresponds to one weight value set, where this weight value set includes multiple weight values.

In some embodiments, the neural network structural layer can be a convolution layer, a pooling layer, a linear rectification layer, a fully connected layer or other type of neural network structure layer. In some embodiments, the classification model111is relative to neural networks (e.g. the classification model111is composed of deep residual networks (ResNet) and fully connected layer, or composed of EfficentNet and fully connected layer).

Reference is further made toFIG. 2, which is a schematic diagram illustrating a machine learning method according to an embodiment of the disclosure. The machine learning device100shown inFIG. 1can be utilized to perform the machine learning method shown inFIG. 2.

As shown inFIG. 2, firstly in step S210, the model parameter MP is obtained from the memory120and the classification model111is performed according to the model parameter MP. In an embodiment, the model parameter MP in the memory120can be obtained according to average values from historical training practices, manual-setting default values, or random values.

In step S220, a first loss and a second loss are calculated according to multiple training samples, where the first loss corresponds to an output layer of the neural network structural layers, and the second loss corresponds to one, which is before the output layer, of the neural network structural layers. In an embodiment, the first loss is generated by the processor110from the output layer of the neural network structural layers of the classification model111, and the second loss is generated by the processor110from the neural network structural layer before the output layer. In some embodiments, the output layer includes at least one fully connection layer. Further details about step S220will be further described in following paragraphs with some examples.

In step S230, multiple updating operations are performed for the model parameter MP according to the first loss and the second loss to train the classification model111. In an embodiment, the model parameter MP is updated by the processor110in the updating operations according to the first loss and the second loss to generate the updated model parameter MP, and the classification model is trained according to the updated model parameter MP to generate the classification model111after the training. Further details about step S230will be further described in following paragraphs with some examples.

By this way, the classification model111after training can be used to execute subsequent applications. For example, the classification model111after the training can be used for object recognition, face recognition, audio recognition, or motion detection within input pictures, images or streaming data, or can be used for data prediction about stock data or weather information.

Reference is further made toFIG. 3andFIG. 4.FIG. 3is a schematic diagram illustrating the classification model and losses according to an embodiment of the disclosure.FIG. 4is a flowchart illustrating further steps S221to S225within step S220in some embodiments.

As shown inFIG. 3, the classification model111includes the neural network structural layers SL1, SL2, . . . SLt. In some embodiments, t is a positive integer. In general, the total quantity of layers in the classification model111can be determined according to application requirements (e.g., classification accuracy requirement, complexity of classification target, and diversity of input images). In some cases, a common range of t can be ranged between 16 and 128, and the disclosure is not limited to a specific quantity of layers.

For example, the neural network structure layers SL1and SL2can be convolutional layers; the neural network structure layer SL3can be a pooling layer; the neural network structure layers SL4and SL5can be convolutional layers; the neural network structure layer SL6can be a pooling layer, the neural network structure layer SL7can be a convolutional layer; the neural network structure layer SL8can be a linear rectification layer; and the neural network structure layer SLt can be a fully connected layer, and the disclosure is not limited thereto.

In some embodiments, the classification model111can have multiple residual mapping blocks, and by using structures of the residual mapping blocks, t can be decreased greatly. The following refers to this structure of the classification model111as examples to further describe step S221to step S224A.

It is added that, for brevity of description, the classification model111inFIG. 3is illustrated as a model with the residual mapping blocks (e.g. ResNet model) for demonstration. The disclosure is not limited thereto. In practical applications, the classification model111may be other type of the convolutional neural networks. In some embodiments, the classification model111is an EfficentNet model.

As shown inFIG. 3andFIG. 4, in step S221, multiple prediction labels {ŷi}i=1nare generated by the processor110from the output layer SLt of the neural network structural layers SL1, SL2, . . . SLt according to the training samples {xi}i=1n. It should be noted that n is a quantity of the training samples {xi}i=1n, n also is a quantity of prediction labels {ŷi}i=1n, n can be a positive integer, and i can be a positive integer which is not more than the quantity n. As shown inFIG. 3, when the training sample Xiis input to the classification model111, the prediction label ŷiis generated from the neural network structural layer SLt (i.e. the output layer) of the classification model111through operations of the neural network structural layers SL1, SL2, . . . SLt. By analogy, the training samples {xi}i=1ncan be input to the classification model111to generate the prediction labels {ŷi}i=1n.

As shown inFIG. 3andFIG. 4, in step S222, the processor110executes a comparison algorithm for comparing the prediction labels {ŷi}i=1nwith multiple training labels {yi}i=1nof the training samples {xi}i=1nto generate the first loss L1. As shown inFIG. 3, the prediction label ŷiis compared with the training label yiof the training sample Xi to calculate a loss. By analogy, multiple losses are calculated by the processor110with comparison algorithms by comparing the prediction labels with training labels, and the first loss L1is generated by the processor110according to these losses (i.e. traditional loss function). In some embodiments, the processor110performs a cross-entropy calculation on the predicted labels {ŷi}i=1nand the training labels {yi}i=1nto obtain the first loss L1.

As shown inFIG. 3andFIG. 4, in step S223, multiple extraction features are generated by the processor110from the classification model111according to the training samples. As shown inFIG. 3, after the training sample Xi is input to the classification model111, the extraction features Hi,1, Hi,2, . . . Hi,mare calculated by artificial neurons of the neural network structural layer Lt-1of the classification model111through the operations of the neural network structural layers SL1, SL2, . . . SLt-1, where m can be a positive integer which is equal to a quantity of the artificial neurons, and the extraction features Hi,1, Hi,2, . . . Hi,mcorresponds to the artificial neurons of the neural network structural layer Lt-1respectively. By analogy, the extraction features {Hi,1, Hi,2, . . . Hi,m}i=1ncorresponding to the training samples {xi}i=1nare calculated from the artificial neurons.

It should be noted that it may exists spurious correlation between the extraction features {Hi,1, Hi,2, . . . Hi,m}i=1nand the training labels {yi}i=1n. In detail, suppose a first extraction feature is causally related to both a second extraction feature and the training label yi, but the second extraction feature and the training label yiare not causally related to each other. Based on this, the second extraction feature and the training label yimay be associated. When the value of the second extraction feature increases along with the change of labels linearly, the second extraction feature is spuriously correlated with the training label yi. The spurious correlation belongs to explicit if the extraction feature which causes the spurious correlation can be observed (i.e. relationship between the first extraction feature, the second extraction feature and the training label yi). Otherwise, the spurious correlation is said to be implicit (i.e. relationship between the second extraction feature and the training label yi). The spurious correlation causes that the predicted labels {ŷi}i=1nare different from the training labels {yi}i=1nmore greatly.

For example, if a patient clinical image usually has a cell tissue of a lesion and a bone which color is similar the cell tissue, it causes the explicit spurious correlation between the extraction feature of the bone and the label of the lesion. For another example, the patient clinical image usually has a background, and the lesion in the patient clinical image is similar to the background. Therefore, it causes the implicit spurious correlation between the extraction feature of the background and the label of the lesion.

To avoid the spurious correlation, the following paragraphs further describes details of using statistical independence to eliminate the explicit spurious correlation and using average treatment effect to eliminate the implicit spurious correlation.

As shown inFIG. 3andFIG. 4, in step S224A, the second loss L2is calculated by the processor110according to statistical independence between the extraction features, where the extraction features {Hi,1, Hi,2, . . . Hi,m}i=1ncorrespond to the one (i.e. the neural network structural layer SLt-1) of the neural network structural layers SL1, SL2, . . . SLt. In detail, statistical independence of random variables is shown in following formula (1).

Where E(.) means an expected value of the random variables, a and b are the random variables, and p and q are positive integers. According to the formula (1), an independent loss can be shown in following formula (2).

As shown inFIG. 3, by replacing the random variables as the extraction features {Hi,1, Hi,2, . . . Hi,m}i=1n, the formula (2) can be rewritten as following formula (3) which indicates the second loss L2(i.e. an independent loss between the extraction features {Hi,1, Hi,2, . . . Hi,m}i=1n).

Where j and k are positive integers and are not more than m. By using the formula (3), the second loss L2is calculated according to the extraction features {Hi,1, Hi,2, . . . Hi,m}i=1n. In some embodiments, the second loss of the formula (3) can further multiply an importance value to generate the second loss L2, where the importance value is more than zero and is a hyperparameter to control importance of the independent loss.

Reference is further made toFIG. 5.FIG. 5is a flowchart illustrating detailed steps S221to S224B within step S220in other embodiments.

It should be noted that difference betweenFIG. 4andFIG. 5is only in step S224B. In other words, in addition to performing step S224A to generate the second loss, alternatively, step S224B can also be performed to generate the second loss. Therefore, the following description is only for step S224B, and the rest of the steps will not be repeated here.

As shown inFIG. 3andFIG. 5, in step S224B, the second loss L3is calculated by the processor110according to according to average treatment effect (ATE) between the extraction features {Hi,1, Hi,2, . . . Hi,m}i=1nand the training labels {yi}i=1nof the training samples {xi}i=1n, where the extraction features {Hi,1, Hi,2, . . . Hi,m}i=1ncorrespond to the one (i.e. the neural network structural layer SLt-1) of the neural network structural layers SL1, SL2, . . . SLt. In detail, average treatment effect (i.e. causality) of random variables is shown in following formula (4).

As shown inFIG. 3, by replacing Yiand Tias the training labels {yi}i=1nand the extraction features {Hi,1, Hi,2, . . . Hi,m}i=1n, the formula (4) can be rewritten as following formula (5).

Where the loss of jth extraction feature means a causal loss (i.e. the average treatment effect loss) corresponding to the extraction features H1,j, H2,j, . . . Hn,j, σ(x) means a hard sigmoid function which is

Based on the formula (5), the second loss L3which indicates the average treatment effect of the {Hi,1, Hi,2, . . . Hi,m}i=1nis shown as following formula (6).

By using the formula (6), the second loss L3is calculated according to the extraction features {Hi,1, Hi,2, . . . Hi,m}i=1nand the training labels {yi}i=1nof the training samples {xi}i=1n. In some embodiments, the second loss of the formula (6) also can further multiply another importance value to generate the second loss L3, where the another importance value is also more than zero and is another hyperparameter to control importance of the average treatment effect loss.

Reference is further made toFIG. 6.FIG. 6is a flowchart illustrating detailed steps S231A to S233within step S230in some embodiments.

As shown inFIG. 6, in step S231A, a loss difference is calculated by the processor110according to the first loss and the second loss. In detail, the processor110performs difference operation between the first loss and the second loss to generate the loss difference (i.e. the first loss subtracts the second loss). It should be noted that the second loss can be generated from step S224A inFIG. 4or step S224B inFIG. 5. In other words, the loss difference can be calculated according to the first loss and the independent loss or according to the first loss and the average treatment effect loss.

In addition, the loss difference also can be calculated according to the first loss, the second loss generated from step S224A inFIG. 4and the second loss generated from step S224B inFIG. 5at the same time (further details will be further described in following paragraphs with some examples).

In step S232, it is to determine whether the loss difference converged. In some embodiments, when the loss difference converged, the loss difference approaches or equals to a difference threshold which is generated according to statistical experiment outcomes.

In this embodiments, if the loss difference did not converge, it performs step S233. In step S233, a backpropagation operation is performed by the processor110for the classification model according to the first loss and the second loss to update the model parameter MP. In other words, an updated model parameter is generated from the model parameter MP according to backpropagation based on the first loss and the second loss.

By this way, it continues to repeat steps s233, S220and S231A for gradually updating the model parameter MP in an iterative manner. Accordingly, the loss difference minimizes gradually (i.e. the second loss maximizes gradually) until the loss difference approaches or equals to the difference threshold. On the contrary, if the loss difference converged, it means that the machine learning device100has completed the training, and the classification model111after training can be used to execute subsequent applications.

Based on aforesaid embodiments, by using the second loss in step S224A, the extraction features belonging to the explicit spurious correlation can be removed in step S230. In addition, by using the second loss in step S224B, the extraction features belonging to the implicit spurious correlation can be removed in step S230.

Reference is further made toFIG. 7.FIG. 7is a flowchart illustrating an additional step after step224A in some embodiments.

As shown inFIG. 7, step S220′A calculates a third loss in same way which calculates the second loss in step S224B. In other words, it means that the processor110generates the independent loss and the average treatment effect loss after generating the first loss. Because step S220′A is similar to step S224B, this step does not repeat here.

Reference is further made toFIG. 8.FIG. 8is a flowchart illustrating a detailed steps S231B to S233within step S230in other embodiments.

It should be noted that difference betweenFIG. 6andFIG. 8is only in step S231B. In other words, in addition to performing step S231A to generate the loss difference, alternatively, step S231B can also be performed to generate the loss difference. Therefore, the following description is only for step S231B, and the rest of the steps will not be repeated here.

As shown asFIG. 8, after step S220′ is performed, step S231B is then performed. In step S231B, a loss difference is calculated by the processor110according to the first loss, the second loss and the third loss. In detail, the processor110performs difference operation between the first loss and the second loss to generate the first difference, and then performs another difference operation between the first difference and the third loss to generate the loss difference (i.e. the first loss subtracts the second loss, and then subtracts the third loss). Therefore, an updated model parameter is generated from the model parameter MP according to backpropagation based on the first loss, the second loss and the third loss in step S233. By this way, it also continues to repeat steps s233, S220and S231B for gradually updating the model parameter MP in an iterative manner. Accordingly, similarly, the loss difference also minimizes gradually (i.e. the second loss and the third loss maximizes gradually) until the loss difference approaches or equals to the difference threshold.

Based on aforesaid embodiments, by using the second loss in step S224A and the third loss in S220′ at the same time, the extraction features belonging to the explicit spurious correlation and the implicit spurious correlation can be removed in step S230.

As shown inFIG. 1, during the training process of the machine learning device100, the model parameter MP of the classification model111is updated according to the first loss and the second loss to avoid the explicit spurious correlation or the implicit spurious correlation between the extraction features and the training labels, where the second loss can be the independent loss or the average treatment effect loss. In addition, by using the independent loss and the average treatment effect loss to adjust the model parameter MP, the explicit spurious correlation and the implicit spurious correlation can be removed, thereby increasing accuracy of prediction of the classification model111greatly.

In the field of computer vision and computer prediction, the accuracy of deep learning mainly relies on a large quantity of labeled training data. As the quality, quantity, and variety of training data increase, the performance of the classification model usually improves correspondingly. However, the classification model always has the explicit spurious correlation or the implicit spurious correlation between the extraction features and the training labels. If we can remove the explicit spurious correlation or the implicit spurious correlation, it will be more efficient and more accurate. In aforesaid embodiments of the disclosure, it proposes adjust the model according to the independent loss and the average treatment effect loss to remove the explicit spurious correlation or the implicit spurious correlation in the classification model. Therefore, the adjusting of the model parameter according to the independent loss and the average treatment effect loss can improve the overall model performance.

For practical applications, the machine learning method and the machine learning device in the disclosure can be utilized in various fields such as machine vision, image classification, data prediction or data classification. For example, this machine learning method can be used in classifying medical images. The machine learning method can be used to classify X-ray images in normal conditions, with pneumonia, with bronchitis, or with heart disease. The machine learning method can also be used to classify ultrasound images with normal fetuses or abnormal fetal positions. The machine learning method can also be used to predict stock data being rising or falling in the future. On the other hand, this machine learning method can also be used to classify images collected in automatic driving, such as distinguishing normal roads, roads with obstacles, and road conditions images of other vehicles. The machine learning method can be utilized in other similar fields. For example, the machine learning methods and machine learning device in the disclosure can also be used in music spectrum recognition, spectral recognition, big data analysis, data feature recognition and other related machine learning fields.