J-MADRAL/P-MADRAL

P-MADRAL is a BERT-sized multi-aspects dense retriever initialized from BERT public checkpoint, further pre-trained on e-commerce product data, and fine-tuned on product search retrieval task on the Amazon ESCI dataset. It uses a symmetric encoder architecture, with a single shared encoder for both queries and products. The similarity function is dot product.

Paper and Repository

P-MADRAL has been described in the Multi-Aspect Joint Retrieval for E-Commerce: Bridging Product Catalogs and Customer Reviews paper. The associated GitHub repository is available at https://anonymous.4open.science/r/J-MADRAL-C4CC.

Usage (HuggingFace Transformers)

Using the model directly in HuggingFace transformers requires additional code available in the repository.

import modeling
import torch
import transformers

# We use a training query from Amazon ESCI as an example.
queries = [
    "iphone 11 pro max case"
]
products = [
    "OtterBox Symmetry Series Case for iPhone 11 Pro Max - Black [...]",
    "Camera Lens Protector for iPhone 11 Pro/Pro Max, Tempered Glass 9H Hardness Anti-Scratch Camera Screen Protective [...]"
]

# Load the tokenizer and model.
tokenizer = transformers.AutoTokenizer.from_pretrained("J-MADRAL/P-MADRAL")
model = modeling.BiEncoderModel.from_pretrained("J-MADRAL/P-MADRAL")

# Tokenize the input data.
q_input = tokenizer(queries,
                    add_special_tokens=True,
                    truncation=True,
                    padding=True,
                    max_length=128,
                    return_tensors="pt")
p_input = tokenizer(products,
                    add_special_tokens=True,
                    truncation=True,
                    padding=True,
                    max_length=128,
                    return_tensors="pt")

# Compute embeddings: take the "pooled_output".
q_emb = model(**q_input).pooled_output
p_emb = model(**p_input).pooled_output

# Compute similarity scores, using dot product similarity.
scores = torch.matmul(q_emb, p_emb.transpose(0, 1))

Training Hyperparameters

Training Stage	Num. Epochs	Learning Rate	AP Scaling Factor	Max Num Tokens	Batch Size	Num Negatives
Pre-training	20	1e-4	0.10	128	64	---
Fine-tuning	20	5e-6	0.05	128	64	7

The data used for fine-tuning is available at https://huggingface.co/datasets/J-MADRAL/TrainingData.

Evaluation Results

Amazon ESCI

Model	R@100	R@500	MRR	nDCG@10	nDCG@50
BM25	0.4949	0.6603	0.4056	0.2599	0.3160
DRAGON	0.5490	0.7155	0.4541	0.2929	0.3540
P-BiBERT	0.6018	0.7640	0.4939	0.3276	0.3967
P-MADRAL	0.6235	0.7806	0.5060	0.3382	0.4104
J-BiBERT	0.5889	0.7556	0.4808	0.3195	0.3858
J-MADRAL	0.6083	0.7729	0.4988	0.3330	0.4028

TREC Product Search 2023

Model	R@100	R@500	MRR	nDCG@10	nDCG@50
BM25	0.7252	0.8650	0.7746	0.6231	0.5988
DRAGON	0.7373	0.8692	0.8135	0.6526	0.6293
P-BiBERT	0.7432	0.8757	0.8256	0.6612	0.6287
P-MADRAL	0.7547	0.8840	0.8337	0.6719	0.6480
J-BiBERT	0.7432	0.8684	0.8114	0.6487	0.6213
J-MADRAL	0.7527	0.8888	0.8333	0.6702	0.6411