Pay Attention to Implicit Attribute Values: A Multi-modal Generative Framework for AVE Task (2023)

Abstract: Attribute Value Extraction (AVE) boosts many e-commerce platform services such as targeted recommendation, product retrieval and question answering. Most previous studies adopt an extractive framework such as named entity recognition (NER) to capture subtokens in the product descriptions as the corresponding values of target attributes. However, in the real world scenario, there also exist implicit attribute values that are not mentioned explicitly but embedded in the image information and implied text meaning of products, for which the power of extractive methods is severely constrained. To address the above issues, we exploit a unified multi-modal AVE framework named DEFLATE (a multi-modal unifieD framEwork For impLicit And expliciT AVE) to acquire implicit attribute values in addition to the explicit ones. DEFLATE consists of a QA-based generation model to produce candidate attribute values from the product information of different modalities, and a discriminative model to ensure the credibility of the generated answers. Meanwhile, to provide a testbed that close to the real world, we collect and annotate a multi-modal dataset with parts of implicit attribute values. Extensive experiments conducted on multiple datasets demonstrate that DEFLATE significantly outperforms previous methods on the extraction of implicit attribute values, while achieving comparable performance for the explicit ones.

## introduction a wide range of e-commerce platforms benefit from accurate annotated product attributes and their values, which could facilitate customers to understand the product better as additional information and help users search for preferred products as a key word @xcite . however, it is a pervasive phenomenon that the manually annotated attribute values of most products on the e-commerce platforms are incomplete and noisy, due to the tedious nature of this work @xcite . to address this, many researches have been proposed for the task of attribute value extraction (ave), aiming at extracting the values of the attributes from the product information such as title or description. current mainstream methods on ave treat it as an information extraction task, the representative models include ner and machine reading comprehension (mrc). the former adopt a set of entity tags for each attribute (e.g., "b-color" and "i-color" for the attribute "color") to identify the corresponding attribute values @xcite @xcite @xcite . while the latter tackles ave by intercepting spans from the product textual information as the values of a given attribute. @xcite . existing works mainly focus on the extraction for the attribute values that are mentioned explicitly in the product descriptions. while there exists significant gap between them and the real world scenario, where an attribute value that needs to be obtained does not usually appear as a subsequence of the product description, but can be inferred from the image, implied text meaning and prior knowledge. fig. 1 shows an intuitive case of this situation, in which the value ("8 inches") corresponding to the attribute of "size" appears in the image in-缤果披萨5选1, 极致享受 benkopizza, choose 1 from 5, ultimate enjoyment 尺⼨: 8寸 size: 8 inches ⻝材: 水果 material: fruits figure 1 : an example of a product with its textual and visual description. the attribute value to be extracted is not a subsequence of product description. inspired by recent progress of text generation paradigms on the field of natural language processing (nlp) @xcite , @xcite explored applying generative models to tackle the ave task. they propose word and positional sequence-based patterns to jointly generate the attributes and the corresponding values. however, this method is still designed to find explicit answers from the textual descriptions in essence, without incorporating the information of images, as well as the semantic information about the attribute names. following this research direction, we further formulate ave as a generative question-and-answer (qa) task, thus empowering the model to predict (generate) the implicit attribute values (iav). specifically, we design a multi-modal unified framework for implicit and explicit ave (called deflate in brief), which takes textual and visual information of products as inputs and adopts an encoder-to-decoder framework. in encoder, taking an attribute as the query, de-flate combines it with the textual description and the image of a product into a single sequence, and a novel multi-modal t5 model is applied to encode the source data incorporating a cross-modal attention mechanism. in decoder, all possible values of a query attribute are generated one by one as the candidate answers. besides, to select the expected attributes, a discriminative model is introduced to determine whether a certain attribute and corresponding values belong to a product or not. moreover, as mentioned above, the existing public ave datasets (such as mave @xcite and aliexpress @xcite ) offer an ideal situation for the extractive methods, in which all attribute values appear explicitly in the textual information. considering the absence of a testbed for the evaluation of implicit ave, we present a multi-modal dataset with a considerable number of iav to support more future related work. our contributions can be summarized as follows: ## approach in this paper, a new framework deflate is introduced for attribute value extraction. we unify the extraction for both explicit and implicit attribute values of products as a multi-modal text generation task. the main idea of deflate is to generate candidate values for each attribute conditioned on the attribute name and the product information first, and then determine the values of which attributes pertain to a certain product using a discriminative model. the overall architecture of deflate is visualised in fig. 2 . ## conclusion in this paper, we pay attention to the implicit attribute values (iav) in ave and propose an effective multi-modal generative-discriminative framework called deflate for attribute value extraction and attribute prediction. other than the attribute values that are mentioned explicitly (eav) in the product descriptions, deflate could also acquire those iav beyond the textual information. we also present a challenging multi-modal dataset for ave with both eav and iav. extensive experiments demonstrate the superiority of deflate over the previous extractive methods on both ave and ap tasks, especially for the products with iav. and our qa-based framework that leverages the relationship between attributes and values on the harder ave of the two tasks shows an advantage over the jointly generative method jg-ave, which utilizes it on the easier ap. besides, the ablation experiments further show the importance of the visual information fusion module and attribute discrimination module in deflate.