Commerce-SessionRec-OTTO-LGBM

🇰🇷 한국어 카드: MODEL_CARD.ko.md

Given the items a shopper has clicked so far in a session, this model returns the 20 items they are most likely to click next. It is trained and evaluated on the OTTO session dataset (a German online retailer, CC BY 4.0) and only knows OTTO item ids.

It is a two-stage recommender. Up to 100 candidates come from repeats of the session's own clicks, two co-visitation tables (click to click, click to cart or order), recent popularity and item2vec neighbours. A LightGBM LambdaRank model re-orders them. The candidates always include the full top-20 of the strong rule-based baseline, so the ranker starts from everything the baseline would show. This is an established recipe, not a new architecture; we claim no research contribution.

Results

Every click position of every evaluation session, ranked over the full catalogue (1,855,603 items), against the same deterministic baseline (repeat + co-visitation + popularity) on the same positions. CIs are paired session-level bootstraps (1,000 resamples). dev has 62% of its next clicks among the 100 candidates; the rest cannot be ranked by any re-ordering. The sealed test split (official OTTO test file, scored once after dev passed) uses the same statistics and ranker.

Split	Metric	This model	Baseline	Difference [95% CI]
dev	Click Recall@20	0.4985	0.4759	+2.26pp [+2.24pp, +2.27pp]
dev	Order Recall@20	0.6840	0.6484	+3.56pp [+3.49pp, +3.63pp]
dev	Click MRR@20	0.2686	0.2382	+3.04pp [+3.02pp, +3.05pp]
test	Click Recall@20	0.4926	0.4701	+2.25pp [+2.23pp, +2.27pp]
test	Order Recall@20	0.6886	0.6532	+3.54pp [+3.46pp, +3.61pp]
test	Click MRR@20	0.2668	0.2366	+3.02pp [+3.00pp, +3.04pp]

Items that were rare in the training data, where our earlier neural model (MultiTRON) failed:

Answer item	This model	Baseline	Difference [95% CI]
dev, answer never clicked in training data	0.4666	0.4513	+1.53pp [+1.46pp, +1.60pp]
dev, answer clicked 1-9 times	0.3023	0.2804	+2.19pp [+2.14pp, +2.24pp]
test, answer never clicked in training data	0.3265	0.3085	+1.81pp [+1.72pp, +1.89pp]
test, answer clicked 1-9 times	0.2844	0.2647	+1.97pp [+1.92pp, +2.02pp]

For reference, MultiTRON (a transformer trained on the same split, not released) reached Click Recall@20 0.4378 and Order Recall@20 0.5720 on dev, below the baseline.

How to use

pip install numpy numba lightgbm huggingface_hub

from huggingface_hub import snapshot_download
import sys
path = snapshot_download("ThakiCloud/Commerce-SessionRec-OTTO-LGBM"); sys.path.insert(0, path)
from sessionrec import SessionRecommender

rec = SessionRecommender(path)                     # loads ranker.txt and stats.npz (777 MiB compressed on disk)
print(rec.recommend([59625, 1253524, 59625], timestamps=[1661200000, 1661200040, 1661200100]))

Pass click timestamps (unix seconds): two ranking features use them. recommend_batch ranks many positions at once. Single-thread CPU latency per request, candidates to ranking: p50 7.6 ms, p95 8.9 ms (2,000 dev positions).

Files

ranker.txt (LightGBM, 862 trees), stats.npz (co-visitation tables, item2vec vectors and neighbours, popularity and item statistics, all computed from OTTO events before 2022-08-21), sessionrec/ (candidate generation, features, ranking), eval_results.json (the run's measured numbers).

How it was trained

Ranker training mirrors the evaluation setup one week earlier: statistics from events before T_dev − 7 days, labels from sessions that start in the following week (3,000,000 sampled positions; 1,847,087 with the next click among the candidates). Relevance is 2 if the next click is later ordered in the session, 1 if it is clicked only. Hyperparameters were fixed in the pre-registration and not tuned on dev; early stopping used a held-out tenth of the training week. Statistics were then rebuilt on all training events (before T_dev) for dev and test. No feature reads the time of the click being predicted.

Limits

• OTTO only. Another shop needs its own statistics and ranker trained on its own logs; this repository does not transfer as is.
• Next-click prediction from clicks. Carts and orders are used as statistics and as order labels, not as the input sequence.
• 38% of dev next clicks are not among the candidates, so no ranker can recommend them.
• Offline evaluation only. No online test has been run.

Data and reproduction

OTTO session dataset, CC BY 4.0 (otto-de/recsys-dataset). The derived statistics in stats.npz carry the same licence and attribution. Pre-registration, code and ledger: commerce-fm (ThakiCloud), docs/decisions/2026-10-24-sessionrec-hybrid-prereg.md, job/rec_otto_hybrid.py, data/ledger/rec_otto_hybrid_1024.json. Training and evaluation ran as one CPU job (32 cores, 5.4 h, no GPU).