Simple and Effective Knowledge-Driven Query Expansion for QA -Based Product Attribute Extraction (2022)

Abstract: A key challenge in attribute value extraction (AVE) from e-commerce sites is how to handle a large number of attributes for diverse products. Although this challenge is partially addressed by a question answering (QA) approach which finds a value in product data for a given query (attribute), it does not work effectively for rare and ambiguous queries. We thus propose simple knowledge-driven query expansion based on possible answers (values) of a query (attribute) for QA-based AVE. We retrieve values of a query (attribute) from the training data to expand the query. We train a model with two tricks, knowledge dropout and knowledge token mixing, which mimic the imperfection of the value knowledge in testing. Experimental results on our cleaned version of AliExpress dataset show that our method improves the performance of AVE (+6.08 macro F1), especially for rare and ambiguous attributes (+7.82 and +6.86 macro F1, respectively).

## introduction one of the most challenging problems in attribute value extraction (ave) from e-commerce sites is a data sparseness problem caused by the diversity of attributes. 1 to alleviate the data sparseness problem, recent researches @xcite formalize the task as question answering (qa) to exploit the similarity of attributes via representation learning. specifically, the qa-based ave takes an attribute name as query and product data as context, and attempts to extract the value from the context. although this approach mitigates the data sparseness problem, performance depends on the quality of query representations @xcite ). 1 aliexpress.com classifies products in the sports & entertainment category using 77,699 attributes @xcite . because attribute names are short and ambiguous as queries, the extraction performance drops significantly for rare attributes with ambiguous names (e.g., sort) which do not represent their values well. aiming to perform more accurate qa-based ave for rare and ambiguous attributes, we propose simple query expansion that exploits values for the attribute as knowledge to learn better query representations (figure 1 , § 3). we first retrieve possible values of each attribute from the training data, and then use the obtained values to augment the query (attribute). since unseen values and attributes will appear in evaluation, we apply dropout to the seen values to mimic the incompleteness of the knowledge ( § 3.2), and perform multi-domain learning to capture the absence of the knowledge ( § 3.3). we demonstrate the effectiveness of the query expansion for bert-based ave model @xcite using the aliexpress dataset foot_0 released by xu et al. (2019) ( § 4). in the evaluation process, we found near-duplicated data in this dataset. we thus construct, from this dataset, a more reliable dataset called cleaned ae-pub to evaluate our method. our contribution is threefold: ## related work attribute value extraction has been modeled as a sequence labeling problem @xcite @xcite @xcite @xcite @xcite @xcite . however, since the number of attributes can exceed ten thousand in e-commerce sites, the models perform poorly for the majority of attributes that rarely appear in the labeled data @xcite . to alleviate the data sparseness problem, xu et al. (2019) introduced a qa-based approach for the ave task. it separately encodes product titles and attributes using bert @xcite and bi-directional long-short term memory @xcite , and then combines the resulting vectors via an attention layer to learn spans of values for the attributes from the titles. @xcite proposed a purely bert-based model, which feeds a string concatenating the given title and attribute to bert. these qa-based ave models, however, do not fully enjoy the advantage of the qa model, since attribute queries are much shorter than sentential questions in the original qa task. to build better queries in solving named entity recognition via qa, @xcite exploited annotation guideline notes for named entity classes as queries. although this approach will be also effective for qa-based ave, it requires substantial labors to prepare manual annotations for more than ten thousand attributes in e-commerce site. ## proposed method this section proposes a simple but effective query expansion method for qa-based ave @xcite by utilizing attribute values. given a product data (title) @xmath0, ..., x n } and an attribute @xmath1, ..., a m }, where n and m denote the number of tokens, the model returns the beginning position, p b , and ending position, p e , of a value. figure 1 depicts the model architecture with our approach. although our query expansion is essentially applicable to any qa-based ave models, we here employ the state-of-the-art model using bert proposed by @xcite . in addition to the qa component for ave, their model has other two components; the no-answer classifier and the distilled masked language model. since those components slightly decrease the overall micro f 1 , we employ the qa component from their model (hearafter, referred to as bert-qa). ## experiments we evaluate our query expansion method for qabased ave on a public dataset, 2 which is built from product data under the sports & entertainment category in aliexpress, following @xcite . ## conclusions we have proposed simple query expansion based on possible values of a given query (attribute) for qabased attribute extraction. with the two tricks to mimic the imperfection of the value knowledge, we retrieve values of given attributes from the training data, and then use the obtained values as knowledge to induce better query representations. experimental results on our cleaned version of the public aliexpress dataset demonstrate that our method improves the performance of product attribute extraction, especially for rare and ambiguous attributes. we will leverage external resources to handle unseen attributes (preliminary experiments are shown in appendix a.4). we will release the script to build our cleaned ae-pub dataset. 6 joe davison, sam shleifer, patrick von platen, clara ma, yacine jernite, julien plu, canwen xu, teven le scao, sylvain gugger, mariama drame, quentin lhoest, and alexander rush. 2020. transformers: state-of-the-art natural language processing. in proceedings of the 2020 conference on empirical methods in natural language processing, pages 38-45, online. association for computational linguistics. huimin xu, wenting wang, xin mao, xinyu jiang, and man lan. 2019. scaling up open tagging from tens to thousands: comprehension empowered attribute value extraction from product title. in proceedings of the 57th annual meeting of the association for computational linguistics, pages 5214-5223, florence, italy. association for computational linguistics. jun yan, nasser zalmout, yan liang, christan grant, xiang ren, and xin luna dong. 2021. adatag: multi-attribute value extraction from product profiles with adaptive decoding. in proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (volume 1: long papers), pages 4694-4705, online. association for computational linguistics. guineng zheng, subhabrata mukherjee, xin luna dong, and feifei li. 2018. opentag: open attribute value extraction from product profiles. in proceedings of the 24th acm sigkdd international conference on knowledge discovery and data mining, pages 1049-1058, london, united kingdom. acm. tiangang zhu, yue wang, haoran li, youzheng wu, xiaodong he, and bowen zhou. 2020. multimodal joint attribute prediction and value extraction for ecommerce product. in proceedings of the 2020 conference on empirical methods in natural language processing (emnlp), pages 2129-2139, online. association for computational linguistics.