E -Commerce Product Categorization with LLM -based Dual-Expert Classification Paradigm (2024)

Abstract: Accurate product categorization in e-commerce is critical for delivering a satisfactory online shopping experience to customers. With the vast number of available products and the numerous potential categories, it becomes crucial to develop a classification system capable of assigning products to their correct categories with high accuracy. We present a dual-expert classification system that utilizes the power of large language models (LLMs). This framework integrates domain-specific knowledge and pre-trained LLM’s general knowledge through effective model fine-tuning and prompting techniques. First, the fine-tuned domain-specific expert recommends top K candidate categories for a given input product. Then, the more general LLM-based expert, through prompting techniques, analyzes the nuanced differences between candidate categories and selects the most suitable target category. We introduce a new in-context learning approach that utilizes LLM self-generated summarization to provide clearer instructions and enhance its performance. Experiments on e-commerce datasets demonstrate the effectiveness of our LLM-based Dual-Expert classification system.

## introduction accurate product categorization on e-commerce sites is the foundation of a structured catalog system to better meet customer needs. a catalog with accurate categorization helps fuel the search engine, which scopes and ranks the search results from queries efficiently. the buyers can find relevant products through the query or browse directly from the targeted categories. the customer behavior can further enhance the downstream personalized tasks like advertisement and item recommendations. eventually the accurate catalog leads to customer satisfaction as well as the revenue. assigning the category for every single product in the world is far from simple. the problem is to map the product description to the label under a well-defined category taxonomy, which includes over thousands of labels. the category selected by the sellers can be noisy due to the vast number of labels and different interpretation of the categories. reviewing and fixing the wrongly assigned items manually is not feasible. therefore, the catalog relies on a categorization system, which utilizes a classification model with high accuracy and coverage to improve the catalog quality. although the classification problems have been researched for years, e-commerce product categorization differs from classical ones. this is due to the vast volume of products with noisy and incomplete signals in both product description and categorical labels. besides, subjective customer opinions about multi-functional products add the complexity, as these opinions can influence product descriptions and optimal category assignment. it is non-trivial to train machine learning models to discern consistent categorical patterns that meet customer expectations for a large population of catalog. we approach product categorization as a text classification problem, since most product items in e-commerce platform are represented through structured or unstructured textual features. recently, pre-trained models (ptms) have shown substantial benefits in capturing universal language representations and strong reasoning capability with rlhf @xcite . two prominent ptm frameworks are: 1) discriminative models with the encoder structure, like bert (bidirectional encoder representations from transformers) @xcite @xcite , and 2) generative models with the decoder structure, like openai's gpt series (generative pre-trained transformer) @xcite @xcite . though some efforts have unified discriminative and generative tasks within a single framework, discriminative language models are generally preferred for sentence understanding, while generative language models are more commonly used for text generation and reasoning. with the increasing parameter sizes and extensive pre-training on vast datasets and various learning tasks, these language models have consistently attained state-of-the-art (sota) performance across numerous nlp benchmarks. given the overlap between pretrained knowledge and e-commerce catalog, we believe that ptms possess the domain knowledge that is necessary to differentiate the nuances between categories. in this study, we introduce an innovative dualexpert framework that integrates both discriminative and generative large language models (llm) in a cascading approach to achieve precise product categorization. initially, the discriminative language model is fine-tuned with domain data, acting as a domain expert to recommend the top k candidate classes for each product. subsequently, an off-the-shelf llm selects the optimal target from the top k suggestions based on certain criteria via prompting. the llm in our framework serves as the general expert due to its capability acquired through pre-training on a large corpus of general data and well alignment with human instructions. the major contribution of this study can be summarized in 3 folds: ## e-commerce product categorization in e-commerce, product categorization involves assigning one or more optimal categories from thousands of labels based on product features. this task is challenging due to noisy and incomplete catalog data. e-commerce sites generally define a taxonomy (a hierarchical structure) as the target label space for categorization. as this taxonomy becomes more granular, categories can become very similar, with only subtle differences distinguishing them. output label space online e-commerce sites pre-define the semantic structure of item categories (known as taxonomy) according to business purpose. this taxonomy serves as the target label space for categorization, and is constructed as hierarchical trees. as the taxonomy tree becomes fine and granular, the categories may appear similar to each other, with only subtle differences separating them. extreme multi-label text classification aims to identify relevant labels from an extremely large set of labels, making it a challenging task @xcite . accuracy of classification models can vary depending on the complexity and dimensionality of the label space. additionally, catalog data inherently suffers from label imbalance, which is widely known as the long tail issue. classification models may struggle to learn patterns for the underrepresented, smaller categories in the skewed distribution. ## llm based dual-expert system llm-based multi-agent systems have emerged as a novel technology with advanced capabilities. these systems specialize llms into various distinct agents, each with different expertise @xcite @xcite . our domain-specific and general expert system has two language models cooperating with each other and each has a specialty. specifically, we have designed two expert models that work sequentially to assign the optimal category to a given product. the whole pipeline is shown in figure 1 . first, a discriminative model work as the domain expert to find top k candidate categories for the selling product given its item data. then, an off-the-shelf llm serves as the general expert, evaluating which categories from the top k candidates are most suitable and accurate for the selling product in question. the llm outputs its decision and the reasoning behind its selection. ## related work when the label space is vast with thousands of labels, a typical approach towards classification based on icl is reducing the label space by identifying most relevant candidates for a given input @xcite . in this regards, research community has worked with both generative and nongenerative techniques to narrow down to most relevant labels. simig et al. (2022) explored generating candidate labels in the setting where task involves classification in unseen labels. zhu and zamani (2023) uses a set of labels and map the llm generated candidates to actual labels by using semantic similarity. d'oosterlinck et al. (2024b) takes a step further and ranks the retrieved labels by using an additional llm. semantic similarity works well when there is direct mapping between input and output. in our work, we target e-commerce data where the direct mapping between input to leaf categories does not work because a large number of leaf categories can have semantically similar definition which defeats the purpose of classifying the product in a single leaf category. further, using multiple llms and making several calls to them is expensive. we reduce that cost by using only one llm that processes the relevant labels selected by a non-generative model. in the non-generative approaches, @xcite considered building an approximate nearest neighbor (ann) graph as an indexing structure over the labels by relying on sparse features engineered from the text. the relevant labels for a given text were then found quickly from the nearest neighbors of the instance via the ann graph. with the introduction of plms, classification performance on several tasks improved significantly through plms' ability of learning better text representation from the raw, unstructured text. in our work, we explore llm's capability for classification in different situations that occur in e-commerce domain -when product text is noisy, and when classification labels are fine-grained and conceptually overlapping. each situation has their own challenges. we show that the dual-expert paradigm overcomes these challenges and outperforms the discriminative classification model in selecting the optimal category. we also show that enhancing the llm prompt with self-generated summarization outperforms other prompt-tuning techniques experimented in this paper. ## conclusion in this study, we propose a dual-expert classification workflow, which leverages the pre-trained llms for accurate e-commerce product categorization. it comprises two experts: a domain-specific expert, trained on a large e-commerce domain data, identifies relevant candidate classes; and a general expert, powered by a llm with in-context learning, that handles nuanced reasoning and decisionmaking. this dual-expert architecture leverages the complementary strengths of each expert, blending specialized domain knowledge with general reasoning capabilities from pre-training, to achieve high classification accuracy in e-commerce categorization.