Extreme Multi-Label Classification with Label Masking for Product Attribute Value Extraction (2022)

Abstract: Although most studies have treated attribute value extraction (AVE) as named entity recognition, these approaches are not practical in real-world e-commerce platforms because they perform poorly, and require canonicalization of extracted values. Furthermore, since values needed for actual services is static in many attributes, extraction of new values is not always necessary. Given the above, we formalize AVE as extreme multi-label classification (XMC). A major problem in solving AVE as XMC is that the distribution between positive and negative labels for products is heavily imbalanced. To mitigate the negative impact derived from such biased distribution, we propose label masking, a simple and effective method to reduce the number of negative labels in training. We exploit attribute taxonomy designed for e-commerce platforms to determine which labels are negative for products. Experimental results using a dataset collected from a Japanese e-commerce platform demonstrate that the label masking improves micro and macro F _1 scores by 3.38 and 23.20 points, respectively.

## introduction since organized product data plays a crucial role in serving better product search and recommendation to customers, attribute value extraction (ave) has become a critical task in the e-commerce industry. although many studies have treated ave as named entity recognition (ner) task ( § 2.1), ner-based approaches are not practical in realworld e-commerce platforms. first, ner-based methods perform poorly because the number of attributes (classes) in e-commerce domains is extremely large @xcite . second, it is necessary to take a further step to normalize extracted values (e.g., coral to pink). to reflect extracted values in actual services, e-commerce platform providers need to convert the values into canonical form by referring their own attribute taxonomy that covers attributes and values for the services. third, extraction of new values is not necessary in many attributes (e.g., country of origin). since it is rare for new values of attributes other than brands to be introduced to the world, it is sufficient to extract the values defined in the attribute taxonomy. given the above reasons, we formalize ave as extreme multi-label classification (xmc), and design a model that directly predicts possible canonical attribute-value pairs except for brands foot_0 from given product data. the main problem in solving ave as xmc is that the number of relevant attribute-value pairs to products is far fewer than that of irrelevant pairs; the majority of attribute-value pairs are regarded as irrelevant (e.g., ⟨memory size, 512gb⟩ for sneakers). to tackle this problem, we propose label masking that mitigates the negative effects of a large amount of irrelevant pairs in training (figure 1 , § 4.2). we detect the irrelevant pairs by referring an attribute taxonomy ( § 3) associated with a real-world dataset we use to train and evaluate models. through experiments using the dataset, we confirm that our label masking method improves micro and macro f 1 scores by 3.38 and 23.20 points, respectively. our contributions can be summarized as follows: ## attribute taxonomy we assume that for each category, attribute taxonomy defines all possible attribute-value pairs that products in the category can take. general attribute-value pairs (e.g., ⟨color, red⟩) are defined for multiple categories. ## proposed method this section proposes our model based on xmc with label masking for the ave task. given a product data @xmath0, t, d⟩, where c denotes a category, t denotes a title consisting of n tokens ({t 1 ,t 2 ,. . . ,t n }) and d denotes a description consisting of m tokens ({d 1 ,d 2 ,. . . ,d m }), respectively, the model returns a set of attribute-value pairs that should be linked with the product data x. figure 1 depicts the model architecture. as a backbone of the architecture, we employ a pretrained bert-base model @xcite , and put a feed forward layer on the top of bert. as an input to bert, we construct a string [cls; t; sep; d] by concatenating t, d, cls and sep; cls and sep are special tokens to represent a classifier token and a separator, respectively. similar with @xcite , we concatenate the last l hidden representations of the cls token, and then feed the concatenated vector into a feed forward category title description attribute-value pairs layer as the representation of the input. the size of the outputs from the feed forward layer is equal to the total number of labels (attributevalue pairs). the outputs are converted into probability through a sigmoid layer, and then pass to the label masking. to mask labels irrelevant to the given product data x, we refer an attribute taxonomy built for an e-commerce platform. we compute binary cross entropy (bce) loss over only relevant labels. in testing, we choose ones whose probability returned from the model exceeds 0.5 among labels relevant to the product data x. ## conclusion in this paper, we formalized ave as xmc, and proposed label masking, a simple and effective method that mitigates the negative impact from the imbalanced distribution of attribute-value pairs relevant and irrelevant to products. experimental results using a real-world dataset show that the label masking improves the performance of bertbased xmc models; it is especially effective for attributes with less training data. as for future work, we plan to see the effectiveness of the label masking method on other tasks in e-commerce domains such as item classification.