t5_recommendation_sports_equipment_english

This model is a fine-tuned version of t5-large on a custom dataset, consisting of sports equipment customers have purchased, and items to recommended next.

This is based on the paper "Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)", where the researchers use a language model as a recommendation system.

• LLMs can "understand" relationships between words/terms via embeddings produced by the transformer architecture. This allows for relationships to be taken into account.
• By feeding an LLM a history of items purchased as the input and the next item purchased as the output, the model can learn what to recommend based on the semantics of the product's name.
  • Taking multiple examples of different users' purchase history into account, the LLM can also learn what genres of products go with what.
  • This essentially replicates collaboritve filtering
• Benefits include:
  • Getting past the cold-start problem with ease (when new items are introduced, the model will be able to understand what's similar to it from the name alone).
  • Avoiding tedious, manual feature engineering (using natural language, the LLM will automatically learn).

The github repository for fine-tuning this model can be viewed here.

The fine-tuned T5 model achieves the following results on the evaluation set:

• Loss: 0.4554
• Rouge1: 57.1429
• Rouge2: 47.6190
• Rougel: 55.5556
• Rougelsum: 55.5556
• Gen Len: 3.9048

Model description

T5 is an open-source sequence-to-sequence model released by Google in 2020, from which several variants have been developed. This fine-tuned version is an attempt to replicate what was presented in the P5 paper, with a custom dataset (based on sports equipment).

More about this model (T5) can be viewed here.

The P5 models from the paper can be viewed on the Hugging Face Hub as well as in this repository.

Intended uses & limitations

Can be used as you please, but is limited to the sports equipment dataset it was fine-tuned on. Your mileage may vary.

Training and evaluation data

Please see this repository for training and evaluation data.

Training procedure

Please see this repository for training and evaluation data.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Framework versions

• Transformers 4.26.0
• Pytorch 1.13.1+cu116
• Datasets 2.8.0
• Tokenizers 0.13.2