Label-Guided Learning for Item Categorization in e-Commerce (2021)

Abstract: Item categorization is an important application of text classification in e-commerce due to its impact on the online shopping experience of users. One class of text classification techniques that has gained attention recently is using the semantic information of the labels to guide the classification task. We have conducted a systematic investigation of the potential benefits of these methods on a real data set from a major e-commerce company in Japan. Furthermore, using a hyperbolic space to embed product labels that are organized in a hierarchical structure led to better performance compared to using a conventional Euclidean space embedding. These findings demonstrate how label-guided learning can improve item categorization systems in the e-commerce domain.

## introduction natural language processing (nlp) techniques have been applied extensively to solve modern ecommerce challenges @xcite . one major nlp challenge in ecommerce is item categorization (ic) which refers to classifying a product based on textual information, typically the product title, into one of numerous categories in the product category taxonomy tree of online stores. although significant progress has been made in the area of text classification, many standard open-source data sets have limited numbers of classes which are not representative of data in industry where there can be hundreds or even thousands of classes @xcite @xcite to cope with the large number of products and the complexity of the category taxonomy, an automated ic system is needed and its prediction quality needs to be high category taxonomy appliance gift men's fashion beverage other... ... coffee tea soft drink chinese tea japanese tea fruit tea root level genre leaf level genre coffee beans instant coffee 伊藤園 お薄抹茶 華の宴(はなのえん) enough to provide positive shopping experiences for customers and consequently drive sales. figure 1 shows an example diagram of the product category taxonomy tree for the ic task. in this example, a tin of japanese tea foot_0 needs to be classified into the leaf level category label "japanese tea." as reviewed in section 2, significant progress has been made on ic as a deep learning text classification task. however, much of the progress in text classification does not make use of the semantic information contained in the labels. recently there have been increasing interest in taking advantage of the semantic information in the labels to improve text classification performance @xcite @xcite @xcite . for the ic task, labels in a product taxonomy tree are actively maintained by human experts and these labels bring rich semantic information. for example, descriptive genre information like "clothes" and "electronics" are used rather than just using a numeric index for the class labels. it is reasonable to surmise that leveraging the semantics of these category labels will improve the ic models. to improve classification performance on several standard text classification data sets, its application to ic on real industry data has been missing thus far. compared to standard data sets, e-commerce data typically contain more complicated label taxonomy tree structures, and product titles tend to be short and do not use standard grammar. therefore, whether label-guided learning can help ic in industry or not is an open question worth investigating. in this paper, we describe our investigation of applying label-guided learning to the ic task. using real data from rakuten 2 , we tested two models: label embedding attentive model (leam) @xcite and label-specific attention network (lsan) @xcite . in addition, to cope with the challenge that labels in an ic task tend to be similar to each other within one product genre, we utilized label embedding methods that can better distinguish labels which led to performance gains. this included testing the use of hyperbolic embeddings which can take into account the hierarchical nature of the taxonomy tree @xcite . the paper is organized as follows: section 2 reviews related research on ic using deep learningbased nlp and the emerging techniques of labelguided learning. section 3 introduces the two label-guided learning models we examined, namely leam and lsan, as well as hyperbolic embedding. section 4 describes experimental results on a large-scale data set from a major e-commerce company in japan. section 5 summarizes our findings and discusses future research directions. ## related works deep learning-based methods have been widely used for the ic task. this includes the use of deep neural network models for item categorization in a hierarchical classifier structure which showed improved performance over conventional machine learning models @xcite , as well as the use of an attention mechanism to identify words that are semantically highly correlated with the predicted categories and therefore can provide improved feature representations for a higher classification performance @xcite . recently, using semantic information carried by label names has received increasing attention in text classification research, and leam @xcite is one of the earliest efforts in this direction 2 https://www.rakuten.co.jp that we are aware of. it uses a joint embedding of both words and class labels to obtain label-specific attention weights to modify the input features. on a set of benchmark text classification data sets, leam showed superior performance over models that did not use label semantics. an extension of leam called lguidedlearn @xcite made further modifications by (a) encoding word inputs first and then using the encoded outputs to compute label attention weights, and (b) using a multihead attention mechanism @xcite to make the attention-weighting mechanism have more representational power. in a related model, lsan @xcite added a label-specific attention branch in addition to a self-attention branch and showed superior performance over models that did not use label semantics on a set of multi-label text classification tasks. alternatively, label names by themselves may not provide sufficient semantic information for accurate text classification. to address this potential shortcoming, longer text can be generated based on class labels to augment the original text input. text generation methods such as using templates and reinforcement learning were compared, and their effectiveness were evaluated using the bert model @xcite with both text sentence and label description as the input @xcite . finally, word embeddings such as word2vec @xcite and glove @xcite are generated in euclidean space. however, embeddings in non-euclidean space called hyperbolic embeddings have been developed @xcite and have been shown to better represent the hierarchical relationship among labels. ## model for a product title x consisting of l words @xmath0, . . . , w l ], our goal is to predict one out of a set of k labels, @xmath1, . . . , c k }. in a neural network-based model, the mapping @xmath2: (a) encoding step (f 0 ), converting x into a numeric tensor representing the input, (b) representation step (f 1 ), processing the input tensor to be a fixed-dimension feature vector z, and (c) classification step (f 2 ), mapping z to y using a feed-forward layer. among label-guided learning models, we chose both leam @xcite and lsan @xcite with respect to the three modeling steps. et al., 2019) for our experiments. ## conclusions since 2018, there have been increasing interest in the field of nlp to use the semantic information of class labels to further improve text classification performance. on the item categorization task in ecommerce, a taxonomy organized in a hierarchical structure already contains rich meaning and provides an ideal opportunity to evaluate the impact of label-guided learning. in this paper, we used real industry data from rakuten, a leading japanese e-commerce platform, to evaluate the benefits of label-guided learning. our experiments showed that lsan is superior to leam because of its usage of context encoding and adaptive combination of both self-and labelattention. we also found that using genre-specific pre-trained embeddings led to better model per-formance than pre-trained embeddings obtained from all product genres. this is likely because pretraining on specific genres allows the embedding to focus on differences between similar genres and the label embeddings are able to take advantage of this. finally, we showed that using hyperbolic embedding, more specifically poincaré embedding, can improve model performance further by ensuring that all class labels are sufficiently separated to allow label-guided learning to work well. one possible limitation of our current work is that although the label embedding is initialized using a hyperbolic embedding, the rest of the training process proceeds in euclidean space. future work could explore the possibility of training the entire model in hyperbolic space. another direction is to incorporate the label-attention mechanism into the bert model @xcite , which has proven to be a powerful approach to text encoding. in addition, more advanced approaches to obtaining better representations of labels on top of our existing approach of using word tokens in labels could be explored.