EAVE : Efficient Product Attribute Value Extraction via Lightweight Sparse-layer Interaction (2024)

Abstract: Product attribute value extraction involves identifying the specific values associated with various attributes from a product profile. While existing methods often prioritize the development of effective models to improve extraction performance, there has been limited emphasis on extraction efficiency. However, in real-world scenarios, products are typically associated with multiple attributes, necessitating multiple extractions to obtain all corresponding values. In this work, we propose an Efficient product Attribute Value Extraction (EAVE) approach via lightweight sparse-layer interaction. Specifically, we employ a heavy encoder to separately encode the product context and attribute. The resulting non-interacting heavy representations of the context can be cached and reused for all attributes. Additionally, we introduce a light encoder to jointly encode the context and the attribute, facilitating lightweight interactions between them. To enrich the interaction within the lightweight encoder, we design a sparse-layer interaction module to fuse the non-interacting heavy representation into the lightweight encoder. Comprehensive evaluation on two benchmarks demonstrate that our method achieves significant efficiency gains with neutral or marginal loss in performance when the context is long and number of attributes is large. Our code is available at: https://anonymous.4open.science/r/EAVE-EA18.

## introduction product attributes serve as crucial features, carrying valuable information about a product. they constitute a fundamental aspect of e-commerce platforms, offering guidance to customers for product comparisons and purchase decisions. additionally, product attributes play a vital role in various applications for merchants, including product recommendations @xcite , search @xcite , and question answering systems @xcite ; figure 1 : an example of product associated with multiple attributes and their corresponding values extracted from the product context. @xcite . attribute value extraction has attracted a lot of attention from both academia and industry, with a plethora of research @xcite @xcite @xcite @xcite being proposed to tackle this problem. with the advancements of transformer models @xcite , attribute value extraction approaches based on transformers @xcite have achieved state-of-the-art performance. these methods concatenate the product attribute and context into a single text sequence and jointly encode it through the self-attention mechanism, effectively capturing the comprehensive interaction between the attribute and context. despite their superior performance, they entail dense computation for extracting each attribute value individually. however, in real-world applications, efficient attribute value extraction is crucial for two main reasons. first, there are millions of new products being generated by the merchants everyday. each product is typically associated with multiple attributes that describe its characteristics from various perspectives. for instance, as illustrated in figure 1 , the 'shoes' has numerous attributes, such as 'brand', 'pattern', 'type', etc., necessitating multiple inferences and resulting in substantial computation costs. second, attribute values are dynamic, undergoing changes such as updates to the product 'price' by the merchant. as a result, re-extraction is required whenever the product profile is updated. hence, efficient extraction poses a significant research problem. several recent efficient sentence pair models @xcite @xcite can be applied to attribute value extraction. these models initially employ a dual encoder architecture to separately encode the product attribute and context. subsequently, they utilize a late interaction layer to combine the attribute and context representations. this approach allows the product context to be cached and utilized for all attributes associated with the product, a concept explored in the context of attribute value extraction as well @xcite . however, these late interaction techniques often yield less effective extraction results due to their neglect of the interaction during the attribute and context encoding. the interaction between the two encoders is particularly crucial in attribute value extraction, especially when the context length is short (additional discussion is presented in the experiments). in such cases, dense interactions become essential to thoroughly capture the connection between the attribute and context. to address these challenges, in this paper, we propose a novel efficient product attribute value extraction (eave) approach via lightweight sparse-layer interaction. in particular, we employ a heavy encoder to separately encode the product context and attribute. the resulting non-interacting heavy representations of the context can be cached and reused for all attributes. moreover, we introduce a light encoder to jointly encode the context and the attribute, enabling lightweight interactions between them. furthermore, we design a sparselayer interaction module to fuse the non-interacting heavy representation into the lightweight encoder, which further enriches the interactions between the context and the context. the evaluations on two product benchmarks demonstrate that our approach achieves similar performance to the state-of-the-art models while being much efficient. we summarize the main contributions as follows: ## main results we present precision, recall, f1, and gflops per example metrics for selected attributes, along with overall results on both mave and ae-110k in ## conclusions efficient product attribute value extraction is an important research problem in many real-world applications. in this work, we proposed an efficient attribute value extraction method with lightweight sparse-layer interaction. specifically, we decouple the computations of context-attribute noninteracting representations and interacting representations, using a heavy and light transformer encoders respectively. additionally, these representations are fused together through the sparse-layer interaction. we conducted benchmarks and systematic ablation studies on two open-sourced prod-uct attribute value extraction datasets. the results demonstrate that our method achieves much fast inference speed while maintaining the performance to the large encoder.