Ask-and-Verify: Span Candidate Generation and Verification for Attribute Value Extraction (2022)

Abstract: The product attribute value extraction (AVE) task aims to capture key factual information from product profiles, and is useful for several downstream applications in e-Commerce platforms. Previous contributions usually formulate this task using sequence labeling or reading comprehension architectures. However, sequence labeling models tend to be conservative in their predictions resulting in a high false negative rate. Existing reading comprehension formulations, on the other hand, can over-generate attribute values which hinders precision. In the present work we address these limitations with a new end-to-end pipeline framework called Ask-and-Verify. Given a product and an attribute query, the Ask step detects the top-K span candidates (i.e. possible attribute values) from the product profiles, then the Verify step filters out false positive candidates. We evaluate Ask-and-Verify model on Amazon’s product pages and AliExpress public dataset, and present a comparative analysis as well as a detailed ablation study. Despite its simplicity, we show that Ask-and-Verify outperforms recent state-of-the-art models by up to 3.1% F1 absolute improvement points, while also scaling to thousands of attributes.

## introduction the product profiles in e-commerce platforms are usually comprised of free-form natural language description of the main product features. the product attribute value extraction (ave) task is used to extract key factual information from textual product descriptions. properly extracted attribute values can facilitate several downstream applications, such as search @xcite , recommendation systems @xcite , and task-oriented dialogue systems @xcite . in the various retail categories there are millions of different product types with thousands of unique attributes, so ave should ideally be scalable with respect to the number of attributes, providing high coverage for all possible values, while maintaining accurate overall predictions. fig. 1 shows an example for the ave task on a dairy product. in this case, ave aims to extract the corresponding attribute values of multiple product attributes including flavor, target age, gluten information, among others. ave is a central organizational task in online shopping systems, significant attention has been paid to the task resulting in a handful of highlyoptimized systems @xcite @xcite @xcite . most of these models use either a sequence labeling formulation, or a machine reading comprehension (mrc) formulation. sequence labeling is a popular formulation in the named entity recognition literature. however, its application on the ave task tends to generate conservative outputs, resulting in many false negatives. this is mostly caused by an overabundance of negative token labels (i.e., the 'o' in bioe schema). recently, the aveqa model @xcite ) addressed the ave task using a reading compre-hension formulation (i.e., question answeringhence aveqa). this formulation tends to be more flexible and scalable than sequential labelling approaches, however, we observed that aveqa tends to over-generate irrelevant outputs and does not generalize to multiple attribute values. in the present work, we address these limitations in existing systems with a new end-to-end framework we dub ask-and-verify, consisting of a span candidate generation step (ask) and a span verification step (verify). the ask step first identifies relevant span candidates by locating potential boundaries (i.e.starting and ending indices) with two individual multi-label classifiers based on token features. the goal of the verify step is to eliminate irrelevant span candidates with span features. the overall framework is attribute-agnostic, which can capture salient attribute information from the input sequence, and can generalize to thousands of attributes without attribute-specific parameters. in summary, we present the ask-and-verify framework, which disentangles the attribute value extraction task into an end-to-end pipeline of (1) span candidate generation and (2) verification. we design the multi-label classifiers and span candidate collection module to obtain valid high-quality span candidates within the model. the verification module is an attribute-agnostic binary classifier based on span features. through extensive experiments on two real-world e-commerce datasets, we show that the ask-and-verify framework outperforms the current crop of state-of-the-art models, and is able to scale to thousands of attributes. ## methodology task definition: given a product description @xmath0, x 2 , ..., x l ] with l tokens, and an attribute a from a pre-defined attribute set a, the ave task aims to extract all of the unique attribute values @xmath1, y 2 , ..., y m } corresponding to a. each attribute value y m is composed of one or more consecutive tokens within x. if no proper attribute values is found in x for a, an empty set should ask-and-verify: our framework addresses the ave task in an end-to-end manner with two major components: (1) span candidate generation (ask), and (2) span candidate verification (verify). for an attribute of interest a, the first step generates the potential span candidates. the second verification step filters the candidates and selects a subset. ## experiments we conduct extensive experiments using the aliexpress @xcite and amazon datasets and compare the ask-and-verify framework with ten state-of-the-art methods. ## deployment considerations the ask-and-verify framework is currently under deployment evaluation. investigation is going on to verify if the framework can be deployed with minimal changes of the existing workflow. in our deployment tests, we found that the ask-and-verify framework has better precision and recall performance. ask-and-verify is also flexible, and can adapt to a wide application scenarios that might require varying precision and recall levels, by changing the threshold values. moreover, a single model can cover a large number of attributes -this is par-ticularly important since e-commerce platforms can hold billions of different products with thousands of attributes. ## conclusions in this paper we described a new end-to-end framework, ask-and-verify, for the attribute value extraction task. this framework has two main components: ( 1 we performed a comparative analysis on an amazon products dataset as well as a publicly available dataset from aliexpress. we evaluate askand-verify compared to ten other baseline models. despite its simplicity, ask-and-verify consistently outperforms these state-of-the-art methods, and is able to scale up to thousands of unique attributes. ask-and-verify also has high flexibility and allows for effective threshold tuning.