AUTOMATIC OPTIMIZATION METHOD AND AUTOMATIC OPTIMIZATION SYSTEM OF DIAGNOSIS MODEL

An automatic optimization method and an automatic optimization system of a diagnosis model are provided. The automatic optimization method includes: obtaining equipment parameters; selecting a target model; selecting and converting a hyperparameter into a gene sequence, randomly generating a plurality of gene sequences to be optimized and adding them to a gene sequence set; performing a gene evolution process to generate a plurality of progeny gene sequences; performing a region search process on the plurality of progeny gene sequences to generate a plurality of new progeny gene sequences and add them to the gene sequence set; and in response to meeting the evolution completion condition, using the gene sequence set as an optimal gene sequence set for configuration of the target model and generation of a plurality of candidate diagnosis models.

CROSS-REFERENCE TO RELATED PATENT APPLICATION

This application claims the benefit of priority to Taiwan Patent Application No. 110132950, filed on Sep. 6, 2021. The entire content of the above identified application is incorporated herein by reference.

FIELD OF THE DISCLOSURE

The present disclosure relates to an optimization method and an optimization system, and more particularly to an automatic optimization method and an automatic optimization system of a diagnosis model.

BACKGROUND OF THE DISCLOSURE

With the development of machine learning technology, industries begin to use machine learning technology to replace human labor, so as to speed up work progress and save labor costs. When a diagnosis model using artificial intelligence is being built, if a diagnosis is to be performed on a manufacturing equipment (such as a die-casting machine), vibration signals can be used to diagnose health of the die-casting machine.

However, pre-processing of data required for building the diagnosis model can only be achieved through cooperation of equipment technicians and data engineers with high professional experience. This process is not only time-consuming, but also requires additional human resources to adjust various hyperparameters in the diagnosis model.

Furthermore, adjustments of the hyperparameters require a large amount of computing resources, and the conventional diagnosis models for evaluating machine health are not capable of simultaneously evaluating computing costs. Therefore, under the condition of limited resources, there is difficulty in performing cost assessments before establishing the diagnosis model.

SUMMARY OF THE DISCLOSURE

In response to the above-referenced technical inadequacies, the present disclosure provides an automatic optimization method and an automatic optimization system of a diagnosis model that can reduce costs of tuning parameters.

In one aspect, the present disclosure provides an automatic optimization method of a diagnosis model, the automatic optimization method includes: obtaining a plurality of equipment parameters of a target equipment; selecting a target model to be used for diagnosing an operation state of the target equipment from a plurality of candidate models, in which the target model has a plurality of hyperparameters; selecting at least one hyperparameter from the plurality of hyperparameters, and converting the at least one hyperparameter that is selected into a gene sequence; according to the at least one hyperparameter and the gene sequence, randomly generating a plurality of gene sequences to be optimized, and adding the plurality of gene sequences to be optimized to a gene sequence set; performing a genetic evolution process to configure the target model with the gene sequence set and perform training, and to optimally select a portion of the gene sequence set according to a training result, so as to breed a plurality progeny gene sequences; performing, for each of the plurality of progeny gene sequences, a region search process to find a plurality neighboring solutions, configure the target model to generate a plurality of models to be searched, train the plurality of models to be searched, and perform an optimal selection to obtain one of a plurality of new progeny gene sequences; adding the plurality of the new progeny gene sequences respectively generated by performing the region search process on the plurality of progeny gene sequences into the gene sequence set; filtering the gene sequence set to obtain a plurality of filtered gene sequences with higher accuracies; determining whether the gene sequence set meets an evolution completion condition, and using the gene sequence set that meets the evolution completion condition as an optimal gene sequence set; and configuring the target model with the optimal gene sequence set to generate a plurality of candidate diagnosis models.

In another aspect, the present disclosure provides an automatic optimization system of a diagnosis model, and the automatic optimization system includes a target equipment and a computing device. The target equipment is configured to generate a plurality of equipment parameters. The computing device includes a processor and a storage, and the processor is configured to obtain the plurality of equipment parameters and store the plurality of equipment parameters in the storage. The processor is configured to: select a target model to be used for diagnosing an operation state of the target equipment from a plurality of candidate models stored in the storage, in which the target model has a plurality of hyperparameters; select at least one hyperparameter from the plurality of hyperparameters, and converting the at least one hyperparameter that is selected into a gene sequence; according to the at least one hyperparameter and the gene sequence, randomly generate a plurality of gene sequences to be optimized, and add the plurality of gene sequences to be optimized to a gene sequence set; perform a genetic evolution process to configure the target model with the gene sequence set and perform training, and to optimally select a portion of the gene sequence set according to a training result, so as to breed a plurality progeny gene sequences; perform, for each of the plurality of progeny gene sequences, a region search process to find a plurality neighboring solutions, configure the target model to generate a plurality of models to be searched, train the plurality of models to be searched, and perform an optimal selection to obtain one of a plurality of new progeny gene sequences; add the plurality of new progeny gene sequences respectively generated by performing the region search process on the plurality of progeny gene sequences into the gene sequence set; filter the gene sequence set to obtain a plurality of filtered gene sequences with higher accuracies; determine whether the gene sequence set meets an evolution completion condition, use the gene sequence set that meets the evolution completion condition as an optimal gene sequence set, and store the optimal gene sequence in the storage; and configure the target model with the optimal gene sequence set to generate a plurality of candidate diagnosis models, and store the plurality of candidate diagnosis models in the storage.

Therefore, in the automatic optimization system and the automatic optimization method of the diagnosis model provided by the present disclosure, the genetic evolution process and the region search mechanism are utilized, which can automatically train artificial intelligence models and can test and verify all possible combinations of all the hyperparameters without spending lots of manpower and costs. Under the condition of limited resources, an optimal hyperparameter combination can be obtained. In addition, dependencies on data scientists can be reduced, and the saved manpower and costs can be used for data insights.

Furthermore, in the automatic optimization system and the automatic optimization method of the diagnosis model provided by the present disclosure, an efficiency frontier line is further generated. According to computing costs and memory limitations, a variety of combinations can be provided for a user's reference, so as to assist the user in selecting the most suitable diagnosis model based on their needs. In this way, cost assessments can be performed before establishing the diagnosis model.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS

First Embodiment

FIG.1is a functional block diagram of an automatic optimization system according to one embodiment of the present disclosure. Reference is made toFIG.1, one embodiment of the present disclosure provides an automatic optimization system1of a diagnosis model, and the automatic optimization system1includes a target equipment10and a computing device12.

The target equipment10can be, for example, a manufacturing equipment, and is configured to generate a plurality of equipment parameters100. The target equipment10can be communicatively connected to the computing device12, so as to transmit the plurality of equipment parameters100to the computing device12through a plurality of equipment signals S1 to Sn. The equipment signals S1 to Sn can be transmitted to the computing device12through interfaces that support various communication protocols, such as an interface that supports open platform communications unified architecture (OPC UA) protocol and/or Modbus protocol. Or, the equipment signals S1 to Sn can be directly transmitted to the computing device12in a form of analog signals. The equipment parameters100can include, for example, a plurality of parameters used in an operation of the target equipment10. If the target equipment10is exemplified as a die casting machine, the equipment parameters100can include, for example, a hydraulic fluid temperature, an injection pressure, a low-speed opening position, a high-speed opening position, a vibration signal, and the like.

Reference is further made toFIG.2, which is a functional block diagram of a computing device according to one embodiment of the present disclosure. Referring toFIG.2, the computing device12can include a processor120, a storage122, a network unit124, a storage unit126, a signal capturing interface127, and an input/output (IO) interface128. The aforementioned components can communicate with one another through, for example, but not limited to, a bus129.

The processor120is electrically coupled to the storage122, and is configured to access computer-readable commands D1 from the storage122, so as to control components in the computing device12to perform functions of the computing device12.

The storage122is any storage device that can be used to store data, such as, but not limited to, a random-access memory (RAM), a read only memory (ROM), a flash memory, a hard disk or other storage devices that can be used to store data. The storage122is configured to at least store the plurality of computer readable instructions D1, a plurality of equipment parameters D2, a plurality of candidate models D0, a gene sequence set D3, an automatic optimization algorithm A0, a genetic algorithm A1, and a region search algorithm A2. In one embodiment, the storage122can also be used to store temporary data generated in response to the processor120performing operations.

The network unit124is configured to access the network under control of the processor120and can, for example, communicate with the target equipment10. Alternatively, the target equipment10can also communicate with the computing device12through the signal capturing interface127. In addition, the signal capturing interface127can, for example, support the aforementioned OPC UA protocol and Modbus protocol, or directly receive analog signals. Accordingly, the equipment signals S1 to Sn can be received, the equipment parameters100can be obtained, and the obtained equipment parameters100can be stored in the storage122.

The storage unit126can be, for example, but not limited to a magnetic disk or an optical disk, so as to store data or instructions under the control of the processor120. The JO interface128can be operated by a user to communicate with the processor120for data input and output.

FIG.3is a flowchart of an automatic optimization method of a diagnosis model according to one embodiment of the present disclosure. InFIG.3, an automatic optimization method of a diagnosis model is provided, which can be applied to the automatic optimization system1shown inFIG.1. The automatic optimization method can also be embodied by other hardware components, such as databases, general processors, computers, servers, unique hardware devices with specific logic circuits, or equipment with specific functions (for example, a unique hardware integrating program codes and a processor/chip). In more detail, the automatic optimization method can be implemented by using a computer program to control components of the automatic optimization system1. The computer program can be stored in a non-transitory computer-readable recording medium, such as a read-only memory, a flash memory, floppy disks, hard disks, optical disks, flash drives, tapes, databases that can be accessed over the Internet, or a computer-readable recording media with the same functions that can be easily achieved by those skilled in the arts.

Referring toFIG.3, after the equipment parameters100of the target equipment10are obtained, the automatic optimization method of the diagnosis model can include configuring the processor120to execute the automatic optimization algorithm A0, so as to perform the following steps.

Step S30: selecting a target model to be used for diagnosing an operation state of the target equipment from a plurality of candidate models. The plurality of candidate models D0 can include, for example, a convolution neural network (CNN) model, an auto-encoder model, a random forest model, a long short-term memory (LSTM) model, or other artificial intelligence models that can be used to monitor the target equipment1for analyzing its operation state with a behavioral process analysis and performing a device state diagnosis.

In the aforementioned artificial intelligence model, there are a plurality of hyperparameters that can be adjusted by the user during a training process, so as to determine a growth curve of the artificial intelligence model. However, conventional ways used to adjust the hyperparameters are complicated and labor intensive, and rely too much on experiences of an adjuster. In general, the hyperparameters used by common artificial intelligence models (also used by the target model) can include, for example, a learning rate, an iteration, a dropout rate, a batch size, a feature selection, an activation function, and the like. In step S305, if the target model is selected as a CNN model, the hyperparameters further include an output size, a kernel size, a stride, a density, a padding and a pooling. In other words, different target models correspond to a plurality of different selectable hyperparameters.

Step S31: selecting at least one hyperparameter from the plurality of hyperparameters, and converting the at least one hyperparameter that is selected into a gene sequence.

Reference can be further made toFIGS.4and5.FIG.4is a flowchart illustrating conversion of a hyperparameter into a gene sequence according to one embodiment of the present disclosure, andFIG.5is a schematic diagram showing the hyperparameter being converted into the gene sequence according to one embodiment of the present disclosure.

As shown inFIG.4, in the aforementioned step S31, the step of converting the at least one hyperparameter into the gene sequence includes:

In response to determining that the at least one hyperparameter is the numerical parameter, step S41is performed. Step S41: encoding, based on a positional notation, a value of the at least one hyperparameter as a part of the gene sequence corresponding to the at least one hyperparameter.

As shown inFIG.5, when the target model is exemplified as a CNN model, the hyperparameters (such as the output size, the core size, the stride, and the density) are determined as the numerical parameters, and numerical values of the above hyperparameters can be encoded in binary form to generate a gene sequence50. However, the present disclosure is not limited thereto. Other positional notations and other encoding methods can also be used to generate the gene sequence50.

In response to determining that the at least one hyperparameter is the categorical parameter, step S42is performed. Step S42: categorically encoding, according to a number of categories of the at least one hyperparameter, the at least one hyperparameter as a part of the gene sequence corresponding to the at least one hyperparameter.

Similarly, as shown inFIG.5, when the target model is exemplified as the CNN model, the hyperparameters (such as the activation function, the padding and the pooling) are determined as the categorical parameters. For example, for the activation function, a ReLU function, a softmax function or a tanh function can be used. The ReLU function, the softmax function, or the tanh function is used to generate category codes 0, 1, and 2 according to the number of categories, so as to generate a gene sequence51.

In addition, in step S31, if two or more than two of the hyperparameters are selected from the plurality of hyperparameters, the gene sequences50and51can be merged into a gene sequence52as shown in a lower part ofFIG.5, and an order of the combination thereof is not limited in the present disclosure.

Reference is made toFIG.3again. After step S31is performed, the automatic optimization method proceeds to step S32: according to the at least one hyperparameter and the gene sequence, randomly generating a plurality of gene sequences to be optimized, and adding the plurality of gene sequences to be optimized to the gene sequence set D3.

For example, as shown inFIG.5, after a structure of the gene sequence52is determined, multiple sets of gene sequences (such as gene sequences53,54) can be randomly generated according to the aforementioned encoding manners. The so-called “random generation” can refer to, for example, having any value of the numeric parameter and any categorical code of the categorical parameter randomly generated first, which can then be converted into a gene sequence, or having a specific range directly assigned to randomly generate a combined gene sequence. For example, ten different gene sequences can be randomly generated and added to the gene sequence set D3.

Step S33: performing a genetic evolution process and a region search process. The genetic evolution process configures and trains the target model based on the gene sequence set, and optimally selects, according to a training result, a portion of the gene sequence to breed a plurality of progeny gene sequences. For each of the plurality of progeny gene sequences, the regional search process is used to find a plurality of neighboring solutions, configure the target model to generate a plurality of models to be searched, train the plurality of models to be searched, and perform an optimal selection to obtain one of a plurality of new progeny gene sequences.

Reference is made toFIG.6, which is a flowchart of a genetic evolution process according to one embodiment of the present disclosure. As shown inFIG.6, the genetic evolution process can include configuring the processor120to execute the genetic algorithm A1, so as to perform the following steps.

Step S60: configuring the target model with the gene sequence set to generate a plurality of models to be trained. For example, ten sets of randomly generated gene sequences (including the selected hyperparameters and types or values defined by the gene sequences) can be applied to the target model (e.g., the CNN model).

Step S61: training the plurality of models to be trained with the plurality of equipment parameters, and evaluating the plurality of models to be trained to obtain a plurality of first accuracies in response to meeting a training completion condition.

In this step, the so-called “training” can include dividing the obtained equipment parameters D2 into a training set, a test set, and a verification set. The model can be adjusted based on the training set first, the adjusted model can then perform predictions based on the verification set, and the hyperparameters can be evaluated. The test set is used to evaluate the final model. For a diagnosis model to be used to evaluate an equipment state of the target equipment, training parameters need to include a labeled data set. For example, to generate a diagnosis model for evaluating health of the die casting machine, the data set must include at least the hydraulic fluid temperature, the injection pressure, the low-speed opening position, the high-speed opening position, the vibration signal, and corresponding health evaluation results.

In addition, during the training process of step S61, a plurality of first computing costs and a plurality of first computing times spent in training the plurality of models to be trained are calculated, respectively. The training completion condition is essentially a convergence condition, so as to avoid endless training. For example, whether the training completion condition is met can be determined by separately determining whether the plurality of first computing costs reach a predetermined computing cost, and determining whether the plurality of first computing times reach a predetermined computing time. For example, whether a duration of the training reaches 30 minutes, or whether a computing power of 30 US dollars is used up can be determined. In response to determining that the plurality of first computing costs reach the predetermined computing cost (e.g., 30 US dollars) or the plurality of first computing times reach the predetermined computing time (e.g., 30 minutes), the training completion condition is determined to have been met, and the plurality of first accuracies of the plurality of models to be trained are recorded.

Step S62: breeding, based on a predetermined mutation rate, the portion of the gene sequence set with the higher first accuracies to generate the plurality of progeny gene sequences.

For example, after the training is completed, the aforementioned plurality of first accuracies are ranked from high to low, and the top two gene sequences are bred with a mutation rate of 0.1%.

Reference is made toFIG.7, which is a schematic diagram showing breeding and mutation in the genetic evolution process according to one embodiment of the present disclosure. As shown inFIG.7, after gene sequences70and72with the higher first accuracies are bred, progeny gene sequences700and720are generated. A breeding mode can be, for example, having parts corresponding to different hyperparameters in the gene sequences70and72exchanged and combined. For example, ten numbers on the left correspond to the output size, and one number on the right corresponds to the activation function. If a mutation occurs during the breeding process based on the predetermined mutation rate, for example, if the progeny gene sequence720generated by the breeding has the mutation, one of the numbers in the progeny gene sequence720is modified. As shown inFIG.7, a fourth binary number from the left in the progeny gene sequence720is modified as a mutant gene Ml, which is modified from 1 to 0 to generate a progeny gene sequence720′. If the number is 0, said number can be modified to 1. However, the above are only examples, and manners, numbers, and locations of the mutations of the gene sequence are not limited in the present disclosure.

Reference is further made toFIG.8, which is a flowchart of a region search process according to one embodiment of the present disclosure. As shown inFIG.8, the region search process can include configuring the processor120to execute the region search algorithm A2, so as to perform the following steps.

Step S80: generating the plurality of neighboring solutions according to a current solution of the at least one hyperparameter of a current progeny gene sequence. Reference can be made toFIG.9, which is a schematic diagram of the region search process according to one embodiment of the present disclosure. For example, if the progeny gene sequence720′ ofFIG.7is taken as the current progeny gene sequence inFIG.9, a current solution corresponding to the activation function is 1, and the code thereof corresponds to the softmax function shown inFIG.5. In other words, the neighboring solutions of the current solution are 0 and 2, which correspond to the ReLU function and the tanh function, respectively.

Step S81: substituting the plurality of neighboring solutions for the current solution in the current progeny gene sequence to generate a current gene sequence set. For example, as shown inFIG.9, if the neighboring solutions are known to be 0 and 2, then these neighboring solutions are used to replace the current solution (i.e., 1) in the progeny gene sequence720′, so as to generate progeny gene sequences80and81. The progeny gene sequences720′,80, and81can be further taken as a current gene sequence set800, which includes a plurality of gene sequences to be searched corresponding to the current solution (i.e., 1) and the neighboring solutions (i.e., 0, 2), that is, the progeny gene sequences720′,80and81.

Step S82: configuring the target model with the current gene sequence set to generate the plurality of models to be searched. This step is similar to step S60, and the current gene sequence set (including the progeny gene sequences720′,80,81) is applied to the target model (for example, the CNN model) to generate the plurality of models to be searched.

Step S83: training the plurality of models to be searched with the plurality of equipment parameters, and evaluating the plurality of models to be trained to obtain a plurality of second accuracies in response to meeting the training completion condition.

During the training process of step S83, a plurality of second computing costs and a plurality of second computing times spent in training the plurality of models to be searched are calculated, respectively. For example, whether the training completion condition is met can be determined by separately determining whether the plurality of second computing costs reach the predetermined computing cost, and determining whether the plurality of second computing times reach the predetermined computing time. For example, whether a duration of the training reaches 30 minutes, or whether a computing power of 30 US dollars is used up can be determined. In response to determining that the plurality of second computing costs reach the predetermined computing cost (e.g., 30 US dollars) or the plurality of second computing times reach the predetermined computing time (e.g., 30 minutes), the training completion condition is determined to have been met, and the plurality of second accuracies of the plurality of models to be searched are recorded.

Step S84: using the gene sequence to be searched with the highest second accuracy as the one of the plurality of new progeny gene sequences. In this step, the second accuracies of the plurality of models to be searched are ranked, and the original progeny gene sequence is replaced by that with a maximum accuracy. For example, as shown inFIG.8, if the second accuracy of the progeny gene sequence81is higher than that of the progeny gene sequence720′, the progeny gene sequence81is used as the one of the plurality of new progeny gene sequences.

Reference is made toFIG.3again, and the automatic optimization method further includes the following steps.

Step S34: adding the plurality of new progeny gene sequences respectively generated by performing the region search process on the plurality of progeny gene sequences into the gene sequence set.

Step S35: filtering the gene sequence set to obtain a plurality of filtered gene sequences with higher accuracies. For example, in each round of evolution, only ten gene sequences with the top ten accuracies can be left and enter the next round of evolution.

Step S36: determining whether the gene sequence set meets an evolution completion condition, and using the gene sequence set that meets the evolution completion condition as an optimal gene sequence set.

In detail, the evolution completion condition is also a convergence condition. For example, whether the gene sequence set meets the evolution completion condition can be determined by, after the gene evolution process and the region search process are performed for multiple times, determining whether a number of times of not improving the maximum accuracy achieved by all the progeny gene sequences in the gene sequence set D3 exceeds a predetermined number of times. For example, in response to the maximum accuracy failing to reach a new high in recent ten or more evolutionary generations, the evolution completion condition is determined to have been met, and the gene sequence set D3 at this time is taken as an optimal gene sequence set.

Therefore, in the present disclosure, by utilizing the genetic evolution process and the region search mechanism, artificial intelligence models can be automatically trained, and all possible combinations of all the hyperparameters can be tested and verified without spending lots of manpower and costs. In this way, under the condition of limited resources, an optimal hyperparameter combination can be obtained.

Step S37: configuring the target model with the optimal gene sequence set to generate a plurality of candidate diagnosis models. This step is used to configure the target model based on a plurality of gene sequences in the optimal gene sequence set, which is similar to step S60and will not be reiterated herein. The generated plurality of candidate diagnosis models can be provided to the user for selection.

In addition, a basis for the user's selection needs to be further provided. In other words, the user needs to select the most suitable diagnosis model for their needs based on the computing time or computing cost of the plurality of candidate diagnosis models. Therefore, as shown inFIG.3, the automatic optimization method can optionally further include the following steps.

Step S38: labeling the plurality of candidate diagnosis models based on a plurality of computing times and a plurality of computing costs of the plurality of candidate diagnosis models.

Step S39: filtering the candidate diagnosis models that exceed a predetermined accuracy, and illustrating an efficiency frontier line according to the computing times and the computing cost of the filtered candidate diagnosis models.

Reference is made toFIG.10, which is a graph showing computing times versus computing costs with an efficiency frontier line according to one embodiment of the present disclosure. As shown inFIG.10, in the optimal gene sequence set, the computing times and computing costs (computing power) spent in the training process are recorded for all the candidate diagnosis models corresponding to the gene sequences. Therefore, after the computing times and the computing costs of the candidate diagnosis models are labeled, the efficiency frontier line can be drawn for these candidate diagnosis models with accuracies more than 80%. The above are only examples, and the present disclosure is not limited thereto.

Therefore, according to computing costs and memory limitations, this efficiency frontier line can provide a variety of combinations for the user's reference, so as to assist the user in selecting the most suitable diagnosis model based on their needs. In this way, cost assessments can be performed before establishing the diagnosis model.

In conclusion, in the automatic optimization system and the automatic optimization method of the diagnosis model provided by the present disclosure, the genetic evolution process and the region search mechanism are utilized, which can automatically train artificial intelligence models and can test and verify all possible combinations of all the hyperparameters without spending lots of manpower and costs. Under the condition of limited resources, an optimal hyperparameter combination can be obtained. In addition, dependencies on data scientists can be reduced, and the saved manpower and costs can be used for data insights.

Furthermore, in the automatic optimization system and the automatic optimization method of the diagnosis model provided by the present disclosure, an efficiency frontier line is further generated. According to computing costs and memory limitations, a variety of combinations can be provided for a user's reference, so as to assist the user in selecting the most suitable diagnosis model based on their needs. In this way, cost assessments can be performed before establishing the diagnosis model.