# Retrieval of Injection Molding Industrial Knowledge Graph Based on Transformer and BERT
## 3 Methodology
### Question Understanding
#### 3.2.3 Entity Alignment  
In the field of injection molding, there are often situations where the same concept is expressed in multiple ways, such as \"excessive flow resistance\" and \"flow resistance is too high\" both expressing the same concept. When users search a knowledge graph, if the system cannot find the corresponding node in the graph according to the concept expressed by the subject, the retrieval performance will be reduced.  
In the past, Levenshtein distance was often used as a basis for determining whether two sentences had similar meanings. Levenshtein distance is calculated as the minimum number of operations needed to transform one sentence into another through insertion,  
Figure 8: Architecture of semantic classification model.  
deletion, or substitution. However, there are many cases where Levenshtein distance cannot be used to determine whether two sentences have similar meanings. For example, the entity pairs \"decrease injection speed\" and \"increase injection speed\" have a very low Levenshtein distance but have different meanings, while the entity pairs \"The product is unable to achieve complete filling\" and \"short shot\" have a very high Levenshtein distance but have the same meaning. These examples demonstrate that it is difficult to rely on Levenshtein distance between entity pairs to accomplish entity alignment tasks.  
Therefore, in this study, we utilize the SBERT (Sentence-BERT) model, as employed in our previous research [34], to identify the nodes in the knowledge graph that have similar concepts to the subject of the user's question. Figure 9 shows the architecture of the SBERT model. When determining the similarity of concepts between the subject and nodes, the model extracts text features individually using BERT, processes the features through Mean Pooling, projects them onto a vector space, and calculates their cosine similarity as a basis for judging whether they have the same concept. In the vector space, entities with similar concepts have higher cosine similarity values, and vice versa.  
This study prepares 1500 entity alignment training data, which are composed of the common vocabulary pairs in injection molding and annotated with whether they have similar semantics. Entities with similar semantics are labeled as 1 and 0 otherwise. The model's loss value is determined by the cosine similarity predicted by the model and the mean square error (MSE) of the annotations, and AdamW is used as the model optimizer with a 0.001 learning rate to train the model for 200 epochs. Regarding the judgment of whether the subject and node express the same semantics, we use a cosine similarity threshold of 0.7. If the cosine similarity between the node and subject is higher than the threshold, they are considered to express the same semantics. After training, the model achieves an accuracy of 0.947 on the 150 test data. The entity alignment dataset prepared in this study includes many common injection molding terms. Therefore, the performance of the model proves its ability to accurately distinguish most injection molding entity pairs with similar semantics in the test data, and the model's performance will improve with an increase in the amount of training data in the future.  
Figure 9: Architecture of the SBERT model.