I mplicit AVE : An Open-Source Dataset and Multimodal LLM s Benchmark for Implicit Attribute Value Extraction (2024)

Abstract: Existing datasets for attribute value extraction (AVE) predominantly focus on explicit attribute values while neglecting the implicit ones, lack product images, are often not publicly available, and lack an in-depth human inspection across diverse domains. To address these limitations, we present ImplicitAVE, the first, publicly available multimodal dataset for implicit attribute value extraction. ImplicitAVE, sourced from the MAVE dataset, is carefully curated and expanded to include implicit AVE and multimodality, resulting in a refined dataset of 68k training and 1.6k testing data across five domains. We also explore the application of multimodal large language models (MLLMs) to implicit AVE, establishing a comprehensive benchmark for MLLMs on the ImplicitAVE dataset. Six recent MLLMs with eleven variants are evaluated across diverse settings, revealing that implicit value extraction remains a challenging task for MLLMs. The contributions of this work include the development and release of ImplicitAVE, and the exploration and benchmarking of various MLLMs for implicit AVE, providing valuable insights and potential future research directions. Dataset and code are available at https://github.com/HenryPengZou/ImplicitAVE.

## introduction attribute value extraction (ave) identifies the value of product attributes from the product information, which is critical in e-commerce for product representation, recommendation, and categorization @xcite @xcite @xcite . the attribute values can be categorized into two types: (1) explicit values can be directly found as a segment in the product text @xcite , while (2) implicit values are never mentioned in the text and can only be inferred from the product image, contextual clues, ø product text: to address these issues, we present implicitave, the first publicly available multimodal dataset for implicit attribute value extraction. we initially sourced product text data from the mave dataset @xcite and then curated the data by eliminating unhelpful attributes and redundant or irrelevant values. subsequently, we transformed and 1 338 ## dataset construction we outline our approach to constructing the first open-source multimodal implicit attribute value extraction dataset, implicitave. the dataset construction pipeline is illustrated in figure 2 . it contains four steps: data collection, curation, expansion, and validation. next, we explain them in detail. ## experiment & benchmark in this section, we describe our experiment results evaluating the effectiveness of various mllms and the previous sota method on our implicitave dataset in diverse settings. ## conclusion in this paper, we introduced implicitave, the first publicly accessible multimodal dataset specifically designed for implicit attribute value extraction, aimed at overcoming the limitations of existing datasets focused on explicit attribute values. by carefully curating attribute values and incorporating both implicit attribute values and product images, implicitave comprises 6.8k training instances and 1.6k human re-annotated high-quality evaluation instances across five diverse domains. moreover, we benchmarked the performance of six recent multimodal large language models on it under diverse settings, highlighting the challenges of implicit value extraction. in the future, we plan to further expand our implicitave dataset to include multi-valued attributes and negative instances. 9 346 6 limitation our implicitave dataset does not consider multivalued attributes and negative instances, i.e. "none" as attribute values. we leave this extension as future work. due to computational resource constraints and limited budgets, we did not evaluate open mllms with parameters larger than 13b.