📚 book_recs_embed (Fine-Tuned Bi-Encoder Retriever)

book_recs_embed is a fine-tuned dense retrieval model specialized for natural language book search and semantic discovery. It is based on BAAI/bge-small-en-v1.5 and was fine-tuned using a two-cycle iterative hard-negative mining curriculum over a curated catalog of ~64,000 books from the UCSD Goodreads dataset.

🚀 Key Improvements (Recall@K over Full 64K Book Catalog)

Evaluated against the complete catalog of 64,552 books:

Metric	Pretrained (bge-small-en-v1.5)	Fine-Tuned (book_recs_embed v5)	Absolute Improvement
Recall@1	11.52%	31.32%	+19.80% (~2.7×)
Recall@5	32.80%	63.90%	+31.10% (~1.9×)
Recall@10	38.84%	70.20%	+31.36% (~1.8×)
Recall@50	53.22%	82.12%	+28.90% (~1.5×)

🛠️ Usage

Using Sentence-Transformers (Recommended)

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("kanishshandilya/book_recs_embed")

# Query instruction recommended for BGE retrieval
instruction = "Represent this sentence for searching relevant passages: "
query = instruction + "atmospheric gothic murder mystery set in a stormy coastal village"

books = [
    "Title: The Silent Patient | Genres: Mystery, Thriller | Description: A woman shoots her husband...",
    "Title: Rebecca | Genres: Gothic, Romance, Mystery | Description: A young bride arrives at Manderley...",
    "Title: Dune | Genres: Sci-Fi, Adventure | Description: Paul Atreides travels to Arrakis..."
]

q_emb = model.encode(query, normalize_embeddings=True)
doc_embs = model.encode(books, normalize_embeddings=True)

similarities = q_emb @ doc_embs.T
for book, sim in sorted(zip(books, similarities), key=lambda x: x[1], reverse=True):
    print(f"[{sim:.4f}] {book[:60]}...")

🧠 Training Methodology

• Synthetic Query Generation: ~320,000 queries generated by Qwen 3 8B (5 diverse queries per book).
• Curriculum: Two-cycle iterative hard negative mining (Cycle 1: v3 -> v4, Cycle 2: v4 -> v5).
• Leakage Prevention: Work-cluster splitting ensures all editions, translations, and graphic novels of a book stay strictly in the same split.