prithivida commited on
Commit
5e7cf20
1 Parent(s): ab1228c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -1
README.md CHANGED
@@ -71,10 +71,11 @@ SPLADE models are a fine balance between retrieval effectiveness (quality) and r
71
  *(Pure MLE folks should not conflate efficiency to model inference efficiency. Our main focus is on retrieval efficiency. Hereinafter efficiency is a short hand for retrieval efficiency unless explicitly qualified otherwise. Not that inference efficiency is not important, we will address that subsequently.)*
72
 
73
  **TL;DR of Our attempt & results**
74
- 1. FLOPS tuning: Seperate **Seq lens and Severely restrictive FLOPs schedule and token budget** doc(128) & query(24) NOT 256 unlike Official SPLADE++. Inspired from **SparseEmbed** (but still we need 2 models for query & doc, This is a doc model).
75
  3. Init Weights: Vanilla **bert-base-uncased**. No corpus awarness unlike Official splade++ / ColBERT
76
  4. Yet achieves competitive effectiveness of MRR@10 **37.22** in ID data (& OOD) and a retrieval latency of - **47.27ms**. (multi-threaded) all On **Consumer grade-GPUs** with **only 5 negatives per query**.
77
  4. For Industry setting: Effectiveness on custom domains needs more than just **Trading FLOPS for tiny gains** and The Premise "SPLADE++ are not well suited to mono-cpu retrieval" does not hold.
 
78
 
79
  <img src="./ID.png" width=750 height=650/>
80
 
 
71
  *(Pure MLE folks should not conflate efficiency to model inference efficiency. Our main focus is on retrieval efficiency. Hereinafter efficiency is a short hand for retrieval efficiency unless explicitly qualified otherwise. Not that inference efficiency is not important, we will address that subsequently.)*
72
 
73
  **TL;DR of Our attempt & results**
74
+ 1. FLOPS tuning: Seperate **Seq lens and Severely restrictive FLOPs schedule and token budget** doc(128) & query(24) NOT 256 unlike Official SPLADE++. Inspired from **SparseEmbed**
75
  3. Init Weights: Vanilla **bert-base-uncased**. No corpus awarness unlike Official splade++ / ColBERT
76
  4. Yet achieves competitive effectiveness of MRR@10 **37.22** in ID data (& OOD) and a retrieval latency of - **47.27ms**. (multi-threaded) all On **Consumer grade-GPUs** with **only 5 negatives per query**.
77
  4. For Industry setting: Effectiveness on custom domains needs more than just **Trading FLOPS for tiny gains** and The Premise "SPLADE++ are not well suited to mono-cpu retrieval" does not hold.
78
+ 5. Owing query-time inference latency we still need 2 models one for query & doc, This is a Doc model and Query model will be released soon.
79
 
80
  <img src="./ID.png" width=750 height=650/>
81