Sequential LLM Framework for Fashion Recommendation (2024)

Abstract: The fashion industry is one of the leading domains in the global e-commerce sector, prompting major online retailers to employ recommendation systems for product suggestions and customer convenience. While recommendation systems have been widely studied, most are designed for general e-commerce problems and struggle with the unique challenges of the fashion domain. To address these issues, we propose a sequential fashion recommendation framework that leverages a pre-trained large language model (LLM) enhanced with recommendation-specific prompts. Our framework employs parameter-efficient fine-tuning with extensive fashion data and introduces a novel mix-up-based retrieval technique for translating text into relevant product suggestions. Extensive experiments show our proposed framework significantly enhances fashion recommendation performance.

## introduction in recent years, fashion e-commerce has garnered considerable global attentions from both consumers and investors. by 2023, the u.s. retail fashion ecommerce market is projected to generate revenues exceeding 207 billion u.s. dollars (gelder). one of the primary objectives of e-commerce is to provide a smooth purchase experience for consumers to purchase products they are looking for. to this end, recommendation systems (rs) have become an essential part of many businesses @xcite . while existing fashion recommendation systems @xcite @xcite predominantly incorporate the visual appearance into the traditional recommendation, they often require resource-intensive processes for image collection and training. additionally, they often struggle to capture the evolving nature of user interactions † work done as an intern at amazon. over time. in light of this, there has been a growing interest in sequential recommendation techniques @xcite @xcite . these techniques model historical user interactions as temporally ordered sequences, thereby achieving remarkable efficacy in capturing both short-term and long-term user preferences. while sequential recommendations have succeeded in general e-commerce, the fashion domain poses unique challenges. our analysis of real-world user interactions on amazon fashion highlights key differences: first, the rapid fashion turnover leads to a sparse user-item interaction matrix, intensifying the cold-start problem @xcite . second, extensive purchase comparisons demand sophisticated approaches to capture finegrained user preferences. third, fashion-specific attributes like seasonality, occasion, and holiday trends require specialized modeling. fourth, diverse search queries that reflect explicit user intentions, necessitate novel modeling techniques. beyond these fashion-specific challenges, traditional recommendation contexts often require specialized models tailored to particular scenarios, such as the cold-start problem @xcite ), which will result in a large number of models that are challenging to maintain and scale. to tackle these challenges holistically, we present a sequential fashion recommendation system augmented by a large language model (llm). trained on vast and diverse datasets, llms have a profound understanding of various domains. leveraging their extensive knowledge and commonsense reasoning capabilities @xcite , llms provide a promising solution to generate meaningful recommendations. this is particularly beneficial in overcoming cold start problems and in accurately discerning fine-grained user preferences. additionally, llms could offer a unified framework capable of addressing diverse recommendation tasks. our llm-augmented recommenda-tion system consists of three primary stages. in the first stage, prompt engineering techniques are used to devise specialized prompts that align with recommendation-specific goals, enabling llm to perceive fine-grained user preferences. in the second stage, we adapt parameter-efficient fine-tuning (peft) techniques @xcite to mitigate prohibitively expensive training costs. in the final stage, we utilize predicted product titles and ids to retrieve and rank potential candidate items. we present a mix-up-based retrieval technique that harnesses the strengths of both id and title embeddings. our contributions can be summarized as follows: within this framework, we propose advanced llm enhancement techniques to address the unique challenges for fashion recommendation. for example, he and mcauley (2016b) proposed the visual bayesian personalized ranking (vbpr), which incorporates visual features extracted from product images into matrix factorization frameworks using pre-trained cnns. yin et al. (2019) utilized visual encodings to learn visual compatibility by training a triplet network, where an anchor item is paired with both a compatible and a non-compatible item to learn embeddings that capture visual compatibility. additionally, yu et al. (2021) introduced a deep aesthetic network that extracts aesthetic features from product images, incorporating them into recommendations to model users' preferences for aesthetic appeal. the methods for extracting visual signals have evolved over time. early studies typically used pre-trained cnns for visual encodings @xcite . however, recent works have shifted towards training visual encoders on specialized datasets @xcite or jointly training visual feature extractors and recommendation modules @xcite . while incorporating visual signals is an inspiring direction, it falls outside the scope of and is furthermore orthogonal to our current study, which focuses on leveraging textual data to model user interactions. this choice is driven by the fact that learning effective product representations from images typically requires large datasets to generalize well @xcite , which would introduce significant demands in terms of data collection and computational resources, making it challenging for industrial deployment. ## fashion characteristics fashion-related shopping presents unique characteristics that must be carefully considered when developing rs. we conduct an in-depth analysis on real-user interaction patterns in amazon fashion, and identify the following key characteristics: c1: high turnover of products. the fashion domain is characterized by a rapid turnover of items, introducing a continuous stream of unique new products to platforms. for instance, amazon fashion adds approximately 3 million new purchasable products each month. additionally, the volume of new fashion items significantly exceeds that of other categories, being 3.6 times greater than new electronics and 6.7 times more than new toy products on amazon. this constant influx leads to a notably sparse user-item interaction matrix, gives rise to the cold-start problem @xcite . c2: thorough purchase comparisons. users involved in fashion-related purchases tend to engage in more comprehensive comparisons than those shopping in other categories. for example, the average interaction length for fashion-related purchases is 55% longer than for electronics and 81% longer than for toy-related purchases. these comprehensive comparisons can be attributed to the extensive range of options-colors, styles, and sizes of fashion products. figure 1 we highlight that while other industries may share some characteristics we've identified, the simultaneous presence of all four is unique to the fashion industry. additionally, the importance of each characteristic in fashion differs from other domains. for example, attributes like seasonality, occasion, and trends have a more fine-grained influence on user choice in fashion compared to electronics or consumable products. in fashion, these factors influence not only availability but also social desirability and attractiveness at a given time. ## conclusion in this paper, we propose a sequential fashion recommendation system enhanced by a llm. our method consists of three stages: prompt creation, training and inference, and retrieval and ranking. first, we design specialized prompts that align the model with recommendation-specific goals. second, we conduct efficient training to optimize the model. third, we introduce a novel mix-up-based retrieval strategy that utilizes both id and title embeddings to finalize item recommendations. extensive experiments show our method significantly enhances fashion recommendation performance.