Attribute Value Generation from Product Title using Language Models (2021)

Abstract: Identifying the value of product attribute is essential for many e-commerce functions such as product search and product recommendations. Therefore, identifying attribute values from unstructured product descriptions is a critical undertaking for any e-commerce retailer. What makes this problem challenging is the diversity of product types and their attributes and values. Existing methods have typically employed multiple types of machine learning models, each of which handles specific product types or attribute classes. This has limited their scalability and generalization for large scale real world e-commerce applications. Previous approaches for this task have formulated the attribute value extraction as a Named Entity Recognition (NER) task or a Question Answering (QA) task. In this paper we have presented a generative approach to the attribute value extraction problem using language models. We leverage the large-scale pretraining of the GPT-2 and the T5 text-to-text transformer to create fine-tuned models that can effectively perform this task. We show that a single general model is very effective for this task over a broad set of product attribute values with the open world assumption. Our approach achieves state-of-the-art performance for different attribute classes, which has previously required a diverse set of models.

## introduction product attributes and their values play an important role in e-commerce platforms. there are hundreds of thousands of products sold online and each type of product has a different set of attributes. these attributes help customers search for products, compare the relevant items and purchase the product of their choice. while details of a product can be found both in its title as well as its description, commonly, the title includes important attributes of the product. everyday many new products are added to the product catalogue often with new attributes types and values. however, attribute information is often sparse, noisy and incomplete with missing values. for example, figure 1 shows a product with its description and attribute value pairs available on the website. it contains attribute values for brand name, type etc., but there are missing attributes, such as "dual-coil" for pickup type, "6" for strings etc. given the wide diversity of products and new products constantly emerging, it is important that attribute value extraction works with the open world assumption, i.e., values for the attributes not seen before. earlier work @xcite @xcite for attribute value extraction use a rule based approach with the help of a domain specific seed dictionary to identify the key phrases. other work have formulated this as named entity recognition (ner) problem (putthividhya and hu, 2011; more, 2016). however, these approaches do not work under the open world assumption. more recently, various neural network based approaches have been proposed and applied to sequence tagging model for attribute value extraction. @xcite is the first to apply the bilstm-crf model for sequence tagging. @xcite propose an end-to-end tagging model using bilstm, crf and attention without any dic-tionary or hand-crafted features. most of these approaches create separate models for different attributes. also, for each attribute a, they have one set of tags to denote beginning (b a ) and inside (i a ) of that attribute. hence, these methods are not scalable for large set of attributes and these models can not identify emerging values for unseen attributes. recent works @xcite have set up this task as question answering (qa) task. question answering in machine reading comprehension (mrc) selects a span of text from the given context to answer the question. @xcite considers product title as context, attribute as query, and proposes to find the attribute value using only global set of bio tags. although the sequence tagging models @xcite achieve promising result, they do not work well for discovering new attributes values. in contrast to past extractive or classification based approaches, we have taken a generative approach to identify attribute values. text generation using language models has several applications in real-world tasks such as text-editing, article writing, sentence completion, etc. text infilling aims to fill the missing part of a given sentence. motivated by their success as well as to leverage the large scale pretraining of the language models, we formulate the attribute value extraction as an instance of text infilling task as well as an answer generation task. we utilize infilling by language modeling (ilm) @xcite for the infilling approach and we fine-tune text-to-text transfer transformer (t5) @xcite ) as an answer generation task. we summarize the main contribution of this work as follows: ## problem statement in this section, we formally define the problem of attribute value generation. given a product context @xmath0, w t 2 , ..., w t m ) and its attribue @xmath1, w a 2 , ..., w a n ), our goal is to generate the value @xmath2, w v 2 , ..., w v e ). for example, the context of the product in figure 1 tributes type and fingerboard material. we want to generate the value "electric guitar" for the attribute type and null for the attribute fingerboard material as this attribute is not present in the context. in this work, first, we formulate this problem as a (i) text infilling task and then as an (ii) answer generation task. for text infilling, we combine the context, t , attribute, a, and the value, v , in a sentence as "t . a is v ." where the attribute value v is masked as blank. our objective is to generate the missing span in this sentence to predict this value. let the incomplete sentence be @xmath3, w s 2 , ..., w s p ). our model outputs the best attribute value sequence ṽ by learning the distribution ṽ = p (v | s). in the answer generation approach, our aim is to generate v as the answer, considering t as the context and a as the question.