prithivida commited on
Commit
b2893b9
1 Parent(s): ab4e07e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -13
README.md CHANGED
@@ -29,19 +29,12 @@ SPLADE models are a fine balance between retrieval effectiveness (quality) and r
29
  *(Pure MLE folks should not conflate efficiency to model inference efficiency. Our main focus is on retrieval efficiency. Hereinafter efficiency is a short hand for retrieval efficiency unless explicitly qualified otherwise. Not that inference efficiency is not important, we will address that subsequently.)*
30
 
31
  **TL;DR of Our attempt & results**
32
- 1. FLOPS tuning:
33
- - Seperate **seq len for doc and query** unlike Official SPLADE++.
34
- - **Severely restricive token budget** doc(128) & query(24) NOT 256 unlike Official SPLADE++.
35
- - Idea Inspired from **SparseEmbed** (instead of 2 models for query & doc).
36
- 2. Init Weights: **MLM adapted on MS MARCO corpus**.
37
- 3. Achieves a modest yet competitive effectiveness - **MRR@10 37.22** in ID data (& OOD).
38
- 2. and a retrieval latency of - **47.27ms**. (multi-threaded)
39
- 3. On **mono-GPU** with **only 5 negatives per query**.
40
- 4. For Industry setting
41
- - Effectiveness on custom domains needs more than just **Trading FLOPS for tiny gains**.
42
- - The Premise "SPLADE++ are not well suited to mono-cpu retrieval" does not hold.
43
-
44
- <img src="./ID.png" width=500 height=350/>
45
 
46
  *Note: The paper refers to the best performing models as SPLADE++, hence for consistency we are reusing the same.*
47
 
 
29
  *(Pure MLE folks should not conflate efficiency to model inference efficiency. Our main focus is on retrieval efficiency. Hereinafter efficiency is a short hand for retrieval efficiency unless explicitly qualified otherwise. Not that inference efficiency is not important, we will address that subsequently.)*
30
 
31
  **TL;DR of Our attempt & results**
32
+ 1. FLOPS tuning: Seperate **Seq lens and Severely restrictive token budget** doc(128) & query(24) NOT 256 unlike Official SPLADE++. Inspired from **SparseEmbed** (instead of 2 models for query & doc).
33
+ 3. Init Weights: **MLM adapted on MS MARCO corpus**.
34
+ 4. Achieves a modest yet competitive effectiveness **MRR@10 37.22** in ID data (& OOD) and a retrieval latency of - **47.27ms**. (multi-threaded) all On **mono-GPU** with **only 5 negatives per query**.
35
+ 4. For Industry setting: Effectiveness on custom domains needs more than just **Trading FLOPS for tiny gains** and The Premise "SPLADE++ are not well suited to mono-cpu retrieval" does not hold.
36
+
37
+ <img src="./ID.png" width=550 height=450/>
 
 
 
 
 
 
 
38
 
39
  *Note: The paper refers to the best performing models as SPLADE++, hence for consistency we are reusing the same.*
40