# Second-Order Text Matching Algorithm for Agricultural Text
## 5 Experiments
### Experimental Setup
#### 5.1.1 Data Collation  
These experiments require a train set, a validation set and a test set. The train set is used for training BERT model during fine-tuning. The validation set is used to select the optimal BERT model during fine-tuning, and the test set is used to determine the size of the candidate set and compare the accuracy and efficiency of SOTM algorithm, BERT model and BM25 algorithm.  
To enable BERT to acquire extensive knowledge on agricultural fields and achieve superior accuracy during fine-tuning, a comprehensive database encompassing agricultural crop knowledge needs to be established. Utilizing crawler technology, a significant amount of data is obtained from agricultural forums. Given the presence of numerous noises in this data, data cleaning is performed to eliminate irrelevant and meaningless images, links, comments, and the like. Furthermore, specialized agricultural books, papers, and documents are tapped to extract additional data, which is then integrated, filtered, and constructed into a high-quality database.  
For each piece of data within the database, a text semantically similar to the original data is constructed through techniques such as synonym replacement and noun explanation. This pair of data is subsequently labeled with 1, signifying their semantic similarity. Conversely, for each piece of data in the database, another distinct piece of data is randomly chosen and modified using similar techniques. This pair is labeled with 0, indicating semantic dissimilarity. The text pairs labeled 1 are then divided into three parts in a specific proportion and allocated to the train set, validation set, and test set accordingly. Additionally, the text pairs labeled 0 are split into two parts and added to the train set and validation set, respectively.  
To enhance the realism of the data and improve the effectiveness of fine-tuning and test, 800 text data samples are randomly selected. For each text, a carefully manually constructed similar meaning text is prepared. Each pair of texts is labeled with 1. Subsequently, 400 data samples are allocated to the train set, 200 to the validation set, and 200 to the test set. To ensure that BERT model comprehends the texts thoroughly, in the train set, the positions of \(text_{1}\) and \(text_{2}\) are swapped for all agricultural data texts labeled with 1. Additionally, \(text_{2}\) and \(text_{1}\) are also included in the train set. Furthermore, the modified text and the original text are segmented using Jieba library for all text data pairs labeled with 1. Any overlapping words found in the two texts have one word randomly masked in the modified text for further modification. If no overlapping words exist, no further modification is performed.  
This paper uses an additional LCQMC dataset to make BERT model more adequate for learning [41], which is a large-scale Chinese text matching dataset containing a lot of semantic information. In order to better reflect the accuracy on the agricultural datasets, the validation set and test set in these experiments do not include the data in LCQMC. The data in the validation set and test set is all related texts in the field of agriculture. The counts of different data types in the train set, validation set, and test set are shown in Table 1.  
In this paragraph, we provide additional clarifications and supplementary details regarding the dataset and experimental setup. Given the immense parameter count of BERT model, directly training and fine-tuning it for text matching on vast text data would be prohibitively expensive. Consequently, we describe our methodology, where BERT model is trained to determine whether two sentences are similar, using distinct train and validation sets. Additionally, we have allocated a small test set to allow the model to actually perform the task of text matching, in order to assess the performance of BERT model in this specific application.