Search Query Refinement for J apanese Named Entity Recognition in E -commerce Domain (2024)

Abstract: In the E-Commerce domain, search query refinement reformulates malformed queries into canonicalized forms by preprocessing operations such as “term splitting” and “term merging”. Unfortunately, most relevant research is rather limited to English. In particular, there is a severe lack of study on search query refinement for the Japanese language. Furthermore, no attempt has ever been made to apply refinement methods to data improvement for downstream NLP tasks in real-world scenarios.This paper presents a novel query refinement approach for the Japanese language. Experimental results show that our method achieves significant improvement by 3.5 points through comparison with BERT-CRF as a baseline. Further experiments are also conducted to measure beneficial impact of query refinement on named entity recognition (NER) as the downstream task. Evaluations indicate that the proposed query refinement method contributes to better data quality, leading to performance boost on E-Commerce specific NER tasks by 11.7 points, compared to search query data preprocessed by MeCab, a very popularly adopted Japanese tokenizer.

## introduction modern e-commerce services rely on named entity recognition (ner) to understand customer demands from their input search queries. in general, ner in e-commerce has enjoyed remarkable success with regard to recognizing important attributes such as brand names from search queries. however, it is prone to poor performance upon malformed queries such as "ni ke" and "adidasmask", which unfortunately occurs frequently due to inevitable typos. to canonicalize the malformed terms in user inputs, two fundamental operations are involved: term merging (e.g., "ni ke" → "nike") and term splitting (e.g., "adidasmask" → "adidas mask"). considering the fact that most previous studies experiments on real japanese queries demonstrate that the combined two-stage method outperforms the standalone approaches like dictionary matching and bert-crf ( §4.1). furthermore, we illustrate that query refinement helps improve data quality, which boosts the accuracy of brand recognition, a classic in-demand ner task in the e-commerce domain. solid experiments show that a sequence tagging model trained with refined search queries achieves a better f 1 score by 11.7 points on customized datasets ( §4.2). in brief, the contribution of this paper is twofold. first, a novel idea of japanese query refinement is designed for practical e-commerce applications. second, quantitative analysis is provided to verify how query refinement benefits brand recognition as a real-world use case. ## proposed method we formalize query reformulation as a characterlevel sequence labeling problem. consider query figure 1 displays overview of our two-step query refinement method. input queries are first matched with entries from three separate dictionaries before being fed into the character-based bert-crf @xcite . prepending a dictionary matching step ahead of the model is necessary as search queries are likely to lack context due to the short length making bert-crf alone struggle to perform very well. if prediction from bert-crf is different from one from the dictionary matching, we give precedence to the dictionary matching. ## experiments we evaluate our query refinement method and its impact on ner tasks using real-world search queries obtained from an e-commerce platform in japan. to compute pmi, we use all queries issued in the past two years after tokenizing the queries with mecab with unidic @xcite . as a result, wikidic and qrydic contain 101, 152, 647 and 53, 570, 779 entries, respectively. dataset for query refinement, two japanese annotators help to label bilou chunk tags for each character in 10,209 queries. for ner, one more japanese annotator labels brand tags, the most essential named entity class in real-world busi-ness, among 13,473 queries. the statistics of both datasets are shown in tables 1 and 2 . model we implemented our model using py-torch. model parameters and the values of θ 1 and θ 2 were determined using the development set. the followings are details of the model training for the query refinement task. we run model training on nvidia-v100 gpu (intel(r) xeon(r) gold 6148 cpu @ 2.40ghz). when fine-tuning, we keep the dropout probability at 0.1 and an optimum number of epochs determined in the development set. the initial learning rate is 1e -5 , and the batch size is 32. we used the pre-trained character-bert model on japanese wikipedia foot_1 . the number of dimensions for hidden representation is 768, the number of transformer blocks is 12, the number of self-attention heads is 12, and the total number of parameters for the pre-trained model is 110m . the vocabulary size is 6, 144. the training time was 10 minutes. ## conclusion whereas most existing research about search query refinement is limited to english, this paper designs a novel idea targeting japanese in e-commerce use case scenarios. we combined bert-crf with keyword matching as novel japanese query refinery which outperforms both schemes when used alone. moreover, we verified through experiments that refine queries lead to much better performance on downstream nlp tasks like product brand recognition. this query refinement system is already adopted in the e-commerce company that provided us with the queries.