TX tract: Taxonomy-Aware Knowledge Extraction for Thousands of Product Categories (2020)

Abstract: Extracting structured knowledge from product profiles is crucial for various applications in e-Commerce. State-of-the-art approaches for knowledge extraction were each designed for a single category of product, and thus do not apply to real-life e-Commerce scenarios, which often contain thousands of diverse categories. This paper proposes TXtract, a taxonomy-aware knowledge extraction model that applies to thousands of product categories organized in a hierarchical taxonomy. Through category conditional self-attention and multi-task learning, our approach is both scalable, as it trains a single model for thousands of categories, and effective, as it extracts category-specific attribute values. Experiments on products from a taxonomy with 4,000 categories show that TXtract outperforms state-of-the-art approaches by up to 10% in F1 and 15% in coverage across all categories.

## introduction real-world e-commerce platforms contain billions of products from thousands of different categories, organized in hierarchical taxonomies (see figure 1 ). knowledge about products can be represented in structured form as a catalog of product attributes (e.g., flavor) and their values (e.g., "strawberry"). understanding precise values of product attributes is crucial for many applications including product search, recommendation, and question answering. however, structured attributes in product catalogs are often sparse, leading to unsatisfactory search results and various kinds of defects. thus, it is invaluable if such structured information can be extracted from product profiles such as product titles and descriptions. consider for instance the "ice cream" product of figure 1 . the corresponding title can potentially * work performed during internship at amazon. be used to extract values for attributes, such as "ben & jerry's" for brand, "strawberry cheesecake" for flavor, and "16 oz" for capacity. state-of-the-art approaches for attribute value extraction @xcite @xcite have employed deep learning to capture features of product attributes effectively for the extraction purpose. however, they are all designed without considering the product categories and thus cannot effectively capture the diversity of categories across the product taxonomy. categories can be substantially different in terms of applicable attributes (e.g., a "camera" product should not have flavor), attribute values (e.g., "vitamin" products may have "fruit" flavor but "banana" products should not) and more generally, text patterns used to describe the attribute values (e.g., the phrase "infused with" is commonly followed by a scent value such as "lavender" in "hair care" products but not in "mattresses" products). in this paper, we consider attribute value extraction for real-world hierarchical taxonomies with thousands of product categories, where directly applying previous approaches presents limitations. on the one extreme, ignoring the hierarchical structure of categories in the taxonomy and assuming a single "flat" category for all products does not capture category-specific characteristics and, as we will show in section 5, is not effective. on the other extreme, training a separate deep neural network for each category in the product taxonomy is prohibitively expensive, and can suffer from lack of training data on small categories. to address the limitations of previous approaches under this challenging setting, we propose a framework for category-specific attribute value extraction that is both efficient and effective. our deep neural network, txtract, is taxonomyaware: it leverages the hierarchical taxonomy of product categories and extracts attribute values for a product conditional to its category, such that it automatically associates categories with specific attributes, valid attribute values, and categoryspecific text patterns. txtract is trained on all categories in parallel and thus can be applied even on small categories with limited labels. the key question we need to answer is how to condition deep sequence models on product categories. our experiments suggest that following previous work to append category-specific artificial tokens to the input sequence, or concatenate category embeddings to hidden neural network layers is not adequate. there are two key ideas behind our solution. first, we use the category information as context to generate category-specific token embeddings via conditional self-attention. second, we conduct multi-task training by meanwhile predicting product category from profile texts; this allows us to get token embeddings that are discriminative of the product categories and further improve attribute extraction. multi-task training also makes our extraction model more robust towards wrong category assignment, which occurs often in real e-commerce websites. 1 to the best of our knowledge, txtract is the first deep neural network that has been applied to attribute value extraction for hierarchical taxonomies with thousands of product categories. in particular, we make three contributions. although this work focuses on e-commerce, our approach to leverage taxonomies can be applied to broader domains such as finance, education, and biomedical/clinical research. we leave experiments on these domains for future work. the rest of this paper is organized as follows. section 2 discusses related work. section 3 presents background and formally defines the problem. section 4 presents our solution and section 5 describes experimental results. finally, section 6 concludes and suggests future work. ## background and problem definition we now provide background on open attribute value extraction (section 3.1) and define our problem of focus (section 3.2). ## txtract model: taxonomy-aware knowledge extraction in this paper, we address open attribute value extraction using a taxonomy-aware deep sequence tagging model, txtract. figure 2 shows the model architecture, which contains two key components: attribute value extraction and product category prediction, accounting for the two tasks in multitask training. both components are taxonomy aware, as we describe next in detail. ## experiments we empirically evaluated txtract and compared it with state-of-the-art models and strong baselines for attribute value extraction on 4000 product categories. txtract leads to substantial improvement across all categories, showing the advantages of leveraging the product taxonomy. ## conclusions and future work we present a novel method for large-scale attribute value extraction for products from a taxonomy with thousands of product categories. our proposed model, txtract, is both efficient and effective: it leverages the taxonomy into a deep neural network to improve extraction quality and can extract attribute values on all categories in parallel. txtract significantly outperforms state-of-the-art approaches and strong baselines under a taxonomy with thousands of product categories. interesting future work includes applying our techniques to different taxonomies (e.g., biomedical) and training a model for different attributes.