Judul: Model Plant-Disease Relation Menggunakan Pendekatan
BERT-BiLSTM-CRF dan Transfer Learning
Abstrak: A Plant-Disease Relation (PDR) is a relation extraction sub-task that aims to reveal the relationship between plants and disease entities in a sentence. Research on the PDR was first proposed by Kim et al. in 2019. They developed PDR as a gold-standard corpus by manually labeling the plant and disease entities with help of the experts. The corpus has four classes: treatment of disease, cause of disease, negative, and associate. Subsequently, they proposed a short dependency path-convolutional neural network (SDP-CNN) method to analyze the relationship between plants and disease entities. The SDP algorithm identifies the shortest path to find plant and disease entities while CNN analyzes the relationship between the two entities in the classification. The experimental results show that the model performance achieved an F1 score of 0.764 for imbalanced class data. The SDP-CNN combination effectively extracts local features from dependency paths and efficiently captures syntactic and semantic information. By concentrating on the most relevant parts of the sentences, SDP-CNN can improve the accuracy of relation extraction tasks, particularly in simpler sentences. On the other hand, the SDP-CNN method has several limitations: 1) Does not take class balance into account when building the PDR model; 2) the SDP-CNN method is only optimal for short sentences, but is not effective for complex English; and 3) rely on Experts when building ground truth datasets. Therefore, this research aims to 1) to select the best techniques in oversampling, undersampling, and SMOTE to deal with imbalance classes; 2) to improve the F1 score of plant-disease relation model proposed by Kim et al. (2019) by combine BERT-BiLSTM-CRF algorithm; and 3) to propose a multi-source transfer learning method for automatic labeling for replaced rely on the expert. Meanwhile, the novelty of this research is as follows: 1) BERT-BiLSTM-CRF method to predict plant-disease relation, and 2) multi-source transfer learning method for automatic labeling. We performed preprocessing and applied stop_word to realize the first goal. Then, the text was converted to numeric text using a Count Vectorizer with the addition of n-grams. In both the training and testing stages, we utilized an 80:20 ratio and 5-fold cross validation. We used machine learning classifiers (multinomial naïve bayes, k-nearest neighbour, support vector machine, and random forest) and Deep Learning (LSTM). Additionally, we employed undersampling, oversampling, and the Synthetic Minority Oversampling Technique (SMOTE) to address with unbalanced classes. Overall, experiments were conducted on all classifiers using the eight schemes. The experimental results show that, the LSTM classifier with the undersampling technique is superior to all the machine learning classifiers. To achieve the second goal, we normalized the PDR corpus. Each sentence was arranged in a single line and ended within a period. Next, the tokenisation process used the BERT Tokenizer. In the training stage, the training and testing data were divided in an 80:20 ratio. Hyperparameter tuning was performed using GridSearch to obtain the optimal model configuration. It took approximately 28 hours to obtain optimal parameters. The results of the GridSearch recommendation yielded the following parameters: epoch=40, dropout=0.3, batch_size=64, and num_units=128. To obtain the optimal model, we conducted the experiments ten times
Keyword: automatic labeling, conditional random field, long short-term memory, plant-disease relation, transfer learning